{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_njm7x1u",
    "id": "C9B4CE5020F84A15930798A3CBA61FF5",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# 优化与深度学习"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_7sadb8d",
    "id": "6AFBCEFB67704543864D5CBCE35CBC56",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 优化与估计"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_ehuuem5",
    "id": "9AC6F434E5F6468B87DC7EE4C0FC1F67",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "尽管优化方法可以最小化深度学习中的损失函数值，但本质上优化方法达到的目标与深度学习的目标并不相同。\n",
    "- 优化方法目标：训练集损失函数值\n",
    "- 深度学习目标：测试集损失函数值（泛化性）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "graffitiCellId": "id_nrbl1j1",
    "id": "2118889B79E94671A9F1C120C6050F53",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import sys\n",
    "sys.path.append('..')\n",
    "import d2lzh1981 as d2l\n",
    "from mpl_toolkits import mplot3d # 三维画图\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "graffitiCellId": "id_mhhopvp",
    "id": "9349E70A9A0B46F487C86AF9A00D3002",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x196556d4a48>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\r\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\r\n",
       "<svg height=\"207.83625pt\" version=\"1.1\" viewBox=\"0 0 338.360937 207.83625\" width=\"338.360937pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n",
       " <defs>\r\n",
       "  <style type=\"text/css\">\r\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\r\n",
       "  </style>\r\n",
       " </defs>\r\n",
       " <g id=\"figure_1\">\r\n",
       "  <g id=\"patch_1\">\r\n",
       "   <path d=\"M 0 207.83625 \r\n",
       "L 338.360937 207.83625 \r\n",
       "L 338.360937 0 \r\n",
       "L 0 0 \r\n",
       "z\r\n",
       "\" style=\"fill:none;\"/>\r\n",
       "  </g>\r\n",
       "  <g id=\"axes_1\">\r\n",
       "   <g id=\"patch_2\">\r\n",
       "    <path d=\"M 52.160938 170.28 \r\n",
       "L 331.160937 170.28 \r\n",
       "L 331.160937 7.2 \r\n",
       "L 52.160938 7.2 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_1\">\r\n",
       "    <g id=\"xtick_1\">\r\n",
       "     <g id=\"line2d_1\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L 0 3.5 \r\n",
       "\" id=\"ma7ea6eb638\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"90.46259\" xlink:href=\"#ma7ea6eb638\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_1\">\r\n",
       "      <!-- 0.6 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 31.78125 66.40625 \r\n",
       "Q 24.171875 66.40625 20.328125 58.90625 \r\n",
       "Q 16.5 51.421875 16.5 36.375 \r\n",
       "Q 16.5 21.390625 20.328125 13.890625 \r\n",
       "Q 24.171875 6.390625 31.78125 6.390625 \r\n",
       "Q 39.453125 6.390625 43.28125 13.890625 \r\n",
       "Q 47.125 21.390625 47.125 36.375 \r\n",
       "Q 47.125 51.421875 43.28125 58.90625 \r\n",
       "Q 39.453125 66.40625 31.78125 66.40625 \r\n",
       "z\r\n",
       "M 31.78125 74.21875 \r\n",
       "Q 44.046875 74.21875 50.515625 64.515625 \r\n",
       "Q 56.984375 54.828125 56.984375 36.375 \r\n",
       "Q 56.984375 17.96875 50.515625 8.265625 \r\n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \r\n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \r\n",
       "Q 6.59375 17.96875 6.59375 36.375 \r\n",
       "Q 6.59375 54.828125 13.0625 64.515625 \r\n",
       "Q 19.53125 74.21875 31.78125 74.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-48\"/>\r\n",
       "       <path d=\"M 10.6875 12.40625 \r\n",
       "L 21 12.40625 \r\n",
       "L 21 0 \r\n",
       "L 10.6875 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-46\"/>\r\n",
       "       <path d=\"M 33.015625 40.375 \r\n",
       "Q 26.375 40.375 22.484375 35.828125 \r\n",
       "Q 18.609375 31.296875 18.609375 23.390625 \r\n",
       "Q 18.609375 15.53125 22.484375 10.953125 \r\n",
       "Q 26.375 6.390625 33.015625 6.390625 \r\n",
       "Q 39.65625 6.390625 43.53125 10.953125 \r\n",
       "Q 47.40625 15.53125 47.40625 23.390625 \r\n",
       "Q 47.40625 31.296875 43.53125 35.828125 \r\n",
       "Q 39.65625 40.375 33.015625 40.375 \r\n",
       "z\r\n",
       "M 52.59375 71.296875 \r\n",
       "L 52.59375 62.3125 \r\n",
       "Q 48.875 64.0625 45.09375 64.984375 \r\n",
       "Q 41.3125 65.921875 37.59375 65.921875 \r\n",
       "Q 27.828125 65.921875 22.671875 59.328125 \r\n",
       "Q 17.53125 52.734375 16.796875 39.40625 \r\n",
       "Q 19.671875 43.65625 24.015625 45.921875 \r\n",
       "Q 28.375 48.1875 33.59375 48.1875 \r\n",
       "Q 44.578125 48.1875 50.953125 41.515625 \r\n",
       "Q 57.328125 34.859375 57.328125 23.390625 \r\n",
       "Q 57.328125 12.15625 50.6875 5.359375 \r\n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \r\n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \r\n",
       "Q 6.984375 17.96875 6.984375 36.375 \r\n",
       "Q 6.984375 53.65625 15.1875 63.9375 \r\n",
       "Q 23.390625 74.21875 37.203125 74.21875 \r\n",
       "Q 40.921875 74.21875 44.703125 73.484375 \r\n",
       "Q 48.484375 72.75 52.59375 71.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-54\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(82.511028 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-54\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_2\">\r\n",
       "     <g id=\"line2d_2\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"141.70226\" xlink:href=\"#ma7ea6eb638\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_2\">\r\n",
       "      <!-- 0.8 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 31.78125 34.625 \r\n",
       "Q 24.75 34.625 20.71875 30.859375 \r\n",
       "Q 16.703125 27.09375 16.703125 20.515625 \r\n",
       "Q 16.703125 13.921875 20.71875 10.15625 \r\n",
       "Q 24.75 6.390625 31.78125 6.390625 \r\n",
       "Q 38.8125 6.390625 42.859375 10.171875 \r\n",
       "Q 46.921875 13.96875 46.921875 20.515625 \r\n",
       "Q 46.921875 27.09375 42.890625 30.859375 \r\n",
       "Q 38.875 34.625 31.78125 34.625 \r\n",
       "z\r\n",
       "M 21.921875 38.8125 \r\n",
       "Q 15.578125 40.375 12.03125 44.71875 \r\n",
       "Q 8.5 49.078125 8.5 55.328125 \r\n",
       "Q 8.5 64.0625 14.71875 69.140625 \r\n",
       "Q 20.953125 74.21875 31.78125 74.21875 \r\n",
       "Q 42.671875 74.21875 48.875 69.140625 \r\n",
       "Q 55.078125 64.0625 55.078125 55.328125 \r\n",
       "Q 55.078125 49.078125 51.53125 44.71875 \r\n",
       "Q 48 40.375 41.703125 38.8125 \r\n",
       "Q 48.828125 37.15625 52.796875 32.3125 \r\n",
       "Q 56.78125 27.484375 56.78125 20.515625 \r\n",
       "Q 56.78125 9.90625 50.3125 4.234375 \r\n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \r\n",
       "Q 19.734375 -1.421875 13.25 4.234375 \r\n",
       "Q 6.78125 9.90625 6.78125 20.515625 \r\n",
       "Q 6.78125 27.484375 10.78125 32.3125 \r\n",
       "Q 14.796875 37.15625 21.921875 38.8125 \r\n",
       "z\r\n",
       "M 18.3125 54.390625 \r\n",
       "Q 18.3125 48.734375 21.84375 45.5625 \r\n",
       "Q 25.390625 42.390625 31.78125 42.390625 \r\n",
       "Q 38.140625 42.390625 41.71875 45.5625 \r\n",
       "Q 45.3125 48.734375 45.3125 54.390625 \r\n",
       "Q 45.3125 60.0625 41.71875 63.234375 \r\n",
       "Q 38.140625 66.40625 31.78125 66.40625 \r\n",
       "Q 25.390625 66.40625 21.84375 63.234375 \r\n",
       "Q 18.3125 60.0625 18.3125 54.390625 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-56\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(133.750697 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-56\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_3\">\r\n",
       "     <g id=\"line2d_3\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"192.941929\" xlink:href=\"#ma7ea6eb638\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_3\">\r\n",
       "      <!-- 1.0 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 12.40625 8.296875 \r\n",
       "L 28.515625 8.296875 \r\n",
       "L 28.515625 63.921875 \r\n",
       "L 10.984375 60.40625 \r\n",
       "L 10.984375 69.390625 \r\n",
       "L 28.421875 72.90625 \r\n",
       "L 38.28125 72.90625 \r\n",
       "L 38.28125 8.296875 \r\n",
       "L 54.390625 8.296875 \r\n",
       "L 54.390625 0 \r\n",
       "L 12.40625 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-49\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(184.990367 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_4\">\r\n",
       "     <g id=\"line2d_4\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"244.181599\" xlink:href=\"#ma7ea6eb638\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_4\">\r\n",
       "      <!-- 1.2 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 19.1875 8.296875 \r\n",
       "L 53.609375 8.296875 \r\n",
       "L 53.609375 0 \r\n",
       "L 7.328125 0 \r\n",
       "L 7.328125 8.296875 \r\n",
       "Q 12.9375 14.109375 22.625 23.890625 \r\n",
       "Q 32.328125 33.6875 34.8125 36.53125 \r\n",
       "Q 39.546875 41.84375 41.421875 45.53125 \r\n",
       "Q 43.3125 49.21875 43.3125 52.78125 \r\n",
       "Q 43.3125 58.59375 39.234375 62.25 \r\n",
       "Q 35.15625 65.921875 28.609375 65.921875 \r\n",
       "Q 23.96875 65.921875 18.8125 64.3125 \r\n",
       "Q 13.671875 62.703125 7.8125 59.421875 \r\n",
       "L 7.8125 69.390625 \r\n",
       "Q 13.765625 71.78125 18.9375 73 \r\n",
       "Q 24.125 74.21875 28.421875 74.21875 \r\n",
       "Q 39.75 74.21875 46.484375 68.546875 \r\n",
       "Q 53.21875 62.890625 53.21875 53.421875 \r\n",
       "Q 53.21875 48.921875 51.53125 44.890625 \r\n",
       "Q 49.859375 40.875 45.40625 35.40625 \r\n",
       "Q 44.1875 33.984375 37.640625 27.21875 \r\n",
       "Q 31.109375 20.453125 19.1875 8.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-50\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(236.230036 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_5\">\r\n",
       "     <g id=\"line2d_5\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"295.421268\" xlink:href=\"#ma7ea6eb638\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_5\">\r\n",
       "      <!-- 1.4 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 37.796875 64.3125 \r\n",
       "L 12.890625 25.390625 \r\n",
       "L 37.796875 25.390625 \r\n",
       "z\r\n",
       "M 35.203125 72.90625 \r\n",
       "L 47.609375 72.90625 \r\n",
       "L 47.609375 25.390625 \r\n",
       "L 58.015625 25.390625 \r\n",
       "L 58.015625 17.1875 \r\n",
       "L 47.609375 17.1875 \r\n",
       "L 47.609375 0 \r\n",
       "L 37.796875 0 \r\n",
       "L 37.796875 17.1875 \r\n",
       "L 4.890625 17.1875 \r\n",
       "L 4.890625 26.703125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-52\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(287.469706 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-52\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_6\">\r\n",
       "     <!-- x -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 54.890625 54.6875 \r\n",
       "L 35.109375 28.078125 \r\n",
       "L 55.90625 0 \r\n",
       "L 45.3125 0 \r\n",
       "L 29.390625 21.484375 \r\n",
       "L 13.484375 0 \r\n",
       "L 2.875 0 \r\n",
       "L 24.125 28.609375 \r\n",
       "L 4.6875 54.6875 \r\n",
       "L 15.28125 54.6875 \r\n",
       "L 29.78125 35.203125 \r\n",
       "L 44.28125 54.6875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-120\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(188.701562 198.556563)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_2\">\r\n",
       "    <g id=\"ytick_1\">\r\n",
       "     <g id=\"line2d_6\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L -3.5 0 \r\n",
       "\" id=\"mbdb32c1a79\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#mbdb32c1a79\" y=\"161.86125\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_7\">\r\n",
       "      <!-- −1.2 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.59375 35.5 \r\n",
       "L 73.1875 35.5 \r\n",
       "L 73.1875 27.203125 \r\n",
       "L 10.59375 27.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-8722\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(20.878125 165.660469)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_2\">\r\n",
       "     <g id=\"line2d_7\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#mbdb32c1a79\" y=\"137.31983\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_8\">\r\n",
       "      <!-- −1.0 -->\r\n",
       "      <g transform=\"translate(20.878125 141.119049)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_3\">\r\n",
       "     <g id=\"line2d_8\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#mbdb32c1a79\" y=\"112.778409\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_9\">\r\n",
       "      <!-- −0.8 -->\r\n",
       "      <g transform=\"translate(20.878125 116.577628)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-56\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_4\">\r\n",
       "     <g id=\"line2d_9\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#mbdb32c1a79\" y=\"88.236989\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_10\">\r\n",
       "      <!-- −0.6 -->\r\n",
       "      <g transform=\"translate(20.878125 92.036208)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-54\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_5\">\r\n",
       "     <g id=\"line2d_10\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#mbdb32c1a79\" y=\"63.695568\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_11\">\r\n",
       "      <!-- −0.4 -->\r\n",
       "      <g transform=\"translate(20.878125 67.494787)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-52\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_6\">\r\n",
       "     <g id=\"line2d_11\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#mbdb32c1a79\" y=\"39.154148\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_12\">\r\n",
       "      <!-- −0.2 -->\r\n",
       "      <g transform=\"translate(20.878125 42.953367)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_7\">\r\n",
       "     <g id=\"line2d_12\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.160938\" xlink:href=\"#mbdb32c1a79\" y=\"14.612727\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_13\">\r\n",
       "      <!-- 0.0 -->\r\n",
       "      <g transform=\"translate(29.257812 18.411946)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_14\">\r\n",
       "     <!-- risk -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 41.109375 46.296875 \r\n",
       "Q 39.59375 47.171875 37.8125 47.578125 \r\n",
       "Q 36.03125 48 33.890625 48 \r\n",
       "Q 26.265625 48 22.1875 43.046875 \r\n",
       "Q 18.109375 38.09375 18.109375 28.8125 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 20.953125 51.171875 25.484375 53.578125 \r\n",
       "Q 30.03125 56 36.53125 56 \r\n",
       "Q 37.453125 56 38.578125 55.875 \r\n",
       "Q 39.703125 55.765625 41.0625 55.515625 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-114\"/>\r\n",
       "      <path d=\"M 9.421875 54.6875 \r\n",
       "L 18.40625 54.6875 \r\n",
       "L 18.40625 0 \r\n",
       "L 9.421875 0 \r\n",
       "z\r\n",
       "M 9.421875 75.984375 \r\n",
       "L 18.40625 75.984375 \r\n",
       "L 18.40625 64.59375 \r\n",
       "L 9.421875 64.59375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-105\"/>\r\n",
       "      <path d=\"M 44.28125 53.078125 \r\n",
       "L 44.28125 44.578125 \r\n",
       "Q 40.484375 46.53125 36.375 47.5 \r\n",
       "Q 32.28125 48.484375 27.875 48.484375 \r\n",
       "Q 21.1875 48.484375 17.84375 46.4375 \r\n",
       "Q 14.5 44.390625 14.5 40.28125 \r\n",
       "Q 14.5 37.15625 16.890625 35.375 \r\n",
       "Q 19.28125 33.59375 26.515625 31.984375 \r\n",
       "L 29.59375 31.296875 \r\n",
       "Q 39.15625 29.25 43.1875 25.515625 \r\n",
       "Q 47.21875 21.78125 47.21875 15.09375 \r\n",
       "Q 47.21875 7.46875 41.1875 3.015625 \r\n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \r\n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \r\n",
       "Q 10.6875 0.296875 5.421875 2 \r\n",
       "L 5.421875 11.28125 \r\n",
       "Q 10.40625 8.6875 15.234375 7.390625 \r\n",
       "Q 20.0625 6.109375 24.8125 6.109375 \r\n",
       "Q 31.15625 6.109375 34.5625 8.28125 \r\n",
       "Q 37.984375 10.453125 37.984375 14.40625 \r\n",
       "Q 37.984375 18.0625 35.515625 20.015625 \r\n",
       "Q 33.0625 21.96875 24.703125 23.78125 \r\n",
       "L 21.578125 24.515625 \r\n",
       "Q 13.234375 26.265625 9.515625 29.90625 \r\n",
       "Q 5.8125 33.546875 5.8125 39.890625 \r\n",
       "Q 5.8125 47.609375 11.28125 51.796875 \r\n",
       "Q 16.75 56 26.8125 56 \r\n",
       "Q 31.78125 56 36.171875 55.265625 \r\n",
       "Q 40.578125 54.546875 44.28125 53.078125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-115\"/>\r\n",
       "      <path d=\"M 9.078125 75.984375 \r\n",
       "L 18.109375 75.984375 \r\n",
       "L 18.109375 31.109375 \r\n",
       "L 44.921875 54.6875 \r\n",
       "L 56.390625 54.6875 \r\n",
       "L 27.390625 29.109375 \r\n",
       "L 57.625 0 \r\n",
       "L 45.90625 0 \r\n",
       "L 18.109375 26.703125 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-107\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(14.798438 97.684531)rotate(-90)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "      <use x=\"41.113281\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "      <use x=\"68.896484\" xlink:href=\"#DejaVuSans-115\"/>\r\n",
       "      <use x=\"120.996094\" xlink:href=\"#DejaVuSans-107\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_13\">\r\n",
       "    <path clip-path=\"url(#pd1d762be4b)\" d=\"M 64.842756 14.612727 \r\n",
       "L 67.404739 16.578432 \r\n",
       "L 69.966723 18.619245 \r\n",
       "L 72.528706 20.733039 \r\n",
       "L 75.09069 22.917537 \r\n",
       "L 77.652673 25.170318 \r\n",
       "L 80.214657 27.488817 \r\n",
       "L 82.77664 29.870331 \r\n",
       "L 85.338623 32.312016 \r\n",
       "L 87.900607 34.8109 \r\n",
       "L 90.46259 37.363875 \r\n",
       "L 93.024574 39.967712 \r\n",
       "L 95.586557 42.619056 \r\n",
       "L 98.148541 45.314433 \r\n",
       "L 100.710524 48.050259 \r\n",
       "L 103.272508 50.822836 \r\n",
       "L 105.834491 53.628362 \r\n",
       "L 108.396475 56.462935 \r\n",
       "L 110.958458 59.32256 \r\n",
       "L 113.520442 62.203147 \r\n",
       "L 116.082425 65.100525 \r\n",
       "L 118.644409 68.010442 \r\n",
       "L 121.206392 70.928572 \r\n",
       "L 123.768376 73.850521 \r\n",
       "L 126.330359 76.771833 \r\n",
       "L 128.892342 79.687996 \r\n",
       "L 131.454326 82.594445 \r\n",
       "L 134.016309 85.486573 \r\n",
       "L 136.578293 88.359736 \r\n",
       "L 139.140276 91.209257 \r\n",
       "L 141.70226 94.030432 \r\n",
       "L 144.264243 96.818543 \r\n",
       "L 146.826227 99.568855 \r\n",
       "L 149.38821 102.27663 \r\n",
       "L 151.950194 104.937132 \r\n",
       "L 154.512177 107.545632 \r\n",
       "L 157.074161 110.097414 \r\n",
       "L 159.636144 112.587787 \r\n",
       "L 162.198128 115.012084 \r\n",
       "L 164.760111 117.365678 \r\n",
       "L 167.322095 119.643978 \r\n",
       "L 169.884078 121.842446 \r\n",
       "L 172.446061 123.956598 \r\n",
       "L 175.008045 125.982011 \r\n",
       "L 177.570028 127.914332 \r\n",
       "L 180.132012 129.749283 \r\n",
       "L 182.693995 131.482667 \r\n",
       "L 185.255979 133.110376 \r\n",
       "L 187.817962 134.628396 \r\n",
       "L 190.379946 136.032816 \r\n",
       "L 192.941929 137.31983 \r\n",
       "L 195.503913 138.485747 \r\n",
       "L 198.065896 139.526995 \r\n",
       "L 200.62788 140.440127 \r\n",
       "L 203.189863 141.221829 \r\n",
       "L 205.751847 141.868921 \r\n",
       "L 208.31383 142.378367 \r\n",
       "L 210.875814 142.74728 \r\n",
       "L 213.437797 142.972923 \r\n",
       "L 215.99978 143.05272 \r\n",
       "L 218.561764 142.984256 \r\n",
       "L 221.123747 142.765283 \r\n",
       "L 223.685731 142.393727 \r\n",
       "L 226.247714 141.86769 \r\n",
       "L 228.809698 141.185452 \r\n",
       "L 231.371681 140.345481 \r\n",
       "L 233.933665 139.34643 \r\n",
       "L 236.495648 138.187145 \r\n",
       "L 239.057632 136.866667 \r\n",
       "L 241.619615 135.384234 \r\n",
       "L 244.181599 133.739285 \r\n",
       "L 246.743582 131.931462 \r\n",
       "L 249.305566 129.960613 \r\n",
       "L 251.867549 127.826793 \r\n",
       "L 254.429533 125.530266 \r\n",
       "L 256.991516 123.071508 \r\n",
       "L 259.553499 120.451205 \r\n",
       "L 262.115483 117.670259 \r\n",
       "L 264.677466 114.729784 \r\n",
       "L 267.23945 111.63111 \r\n",
       "L 269.801433 108.37578 \r\n",
       "L 272.363417 104.965553 \r\n",
       "L 274.9254 101.402402 \r\n",
       "L 277.487384 97.688514 \r\n",
       "L 280.049367 93.826288 \r\n",
       "L 282.611351 89.818337 \r\n",
       "L 285.173334 85.667482 \r\n",
       "L 287.735318 81.376755 \r\n",
       "L 290.297301 76.949393 \r\n",
       "L 292.859285 72.38884 \r\n",
       "L 295.421268 67.698739 \r\n",
       "L 297.983252 62.882936 \r\n",
       "L 300.545235 57.945469 \r\n",
       "L 303.107218 52.890574 \r\n",
       "L 305.669202 47.722673 \r\n",
       "L 308.231185 42.446376 \r\n",
       "L 310.793169 37.066473 \r\n",
       "L 313.355152 31.587932 \r\n",
       "L 315.917136 26.015895 \r\n",
       "L 318.479119 20.355669 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_14\">\r\n",
       "    <path clip-path=\"url(#pd1d762be4b)\" d=\"M 64.842756 14.612727 \r\n",
       "L 67.404739 20.417556 \r\n",
       "L 69.966723 26.202961 \r\n",
       "L 72.528706 31.874611 \r\n",
       "L 75.09069 37.342622 \r\n",
       "L 77.652673 42.523723 \r\n",
       "L 80.214657 47.343244 \r\n",
       "L 82.77664 51.736896 \r\n",
       "L 85.338623 55.652294 \r\n",
       "L 87.900607 59.050174 \r\n",
       "L 90.46259 61.905296 \r\n",
       "L 93.024574 64.206987 \r\n",
       "L 95.586557 65.959333 \r\n",
       "L 98.148541 67.180999 \r\n",
       "L 100.710524 67.904685 \r\n",
       "L 103.272508 68.17624 \r\n",
       "L 105.834491 68.053447 \r\n",
       "L 108.396475 67.604507 \r\n",
       "L 110.958458 66.906276 \r\n",
       "L 113.520442 66.042271 \r\n",
       "L 116.082425 65.100525 \r\n",
       "L 118.644409 64.171318 \r\n",
       "L 121.206392 63.344856 \r\n",
       "L 123.768376 62.708949 \r\n",
       "L 126.330359 62.346748 \r\n",
       "L 128.892342 62.334591 \r\n",
       "L 131.454326 62.740018 \r\n",
       "L 134.016309 63.620008 \r\n",
       "L 136.578293 65.019458 \r\n",
       "L 139.140276 66.969982 \r\n",
       "L 141.70226 69.489012 \r\n",
       "L 144.264243 72.579268 \r\n",
       "L 146.826227 76.228577 \r\n",
       "L 149.38821 80.410064 \r\n",
       "L 151.950194 85.082706 \r\n",
       "L 154.512177 90.192227 \r\n",
       "L 157.074161 95.672329 \r\n",
       "L 159.636144 101.446215 \r\n",
       "L 162.198128 107.428368 \r\n",
       "L 164.760111 113.526554 \r\n",
       "L 167.322095 119.643978 \r\n",
       "L 169.884078 125.68157 \r\n",
       "L 172.446061 131.540314 \r\n",
       "L 175.008045 137.123583 \r\n",
       "L 177.570028 142.339417 \r\n",
       "L 180.132012 147.102688 \r\n",
       "L 182.693995 151.337093 \r\n",
       "L 185.255979 154.976942 \r\n",
       "L 187.817962 157.968674 \r\n",
       "L 190.379946 160.272091 \r\n",
       "L 192.941929 161.86125 \r\n",
       "L 195.503913 162.725022 \r\n",
       "L 198.065896 162.867273 \r\n",
       "L 200.62788 162.306693 \r\n",
       "L 203.189863 161.076255 \r\n",
       "L 205.751847 159.222325 \r\n",
       "L 208.31383 156.803452 \r\n",
       "L 210.875814 153.888852 \r\n",
       "L 213.437797 150.556639 \r\n",
       "L 215.99978 146.891844 \r\n",
       "L 218.561764 142.984256 \r\n",
       "L 221.123747 138.926159 \r\n",
       "L 223.685731 134.810011 \r\n",
       "L 226.247714 130.726118 \r\n",
       "L 228.809698 126.760367 \r\n",
       "L 231.371681 122.992076 \r\n",
       "L 233.933665 119.492003 \r\n",
       "L 236.495648 116.320579 \r\n",
       "L 239.057632 113.526389 \r\n",
       "L 241.619615 111.144959 \r\n",
       "L 244.181599 109.197864 \r\n",
       "L 246.743582 107.692187 \r\n",
       "L 249.305566 106.620335 \r\n",
       "L 251.867549 105.960227 \r\n",
       "L 254.429533 105.67584 \r\n",
       "L 256.991516 105.718103 \r\n",
       "L 259.553499 106.02612 \r\n",
       "L 262.115483 106.528687 \r\n",
       "L 264.677466 107.146068 \r\n",
       "L 267.23945 107.791986 \r\n",
       "L 269.801433 108.37578 \r\n",
       "L 272.363417 108.804677 \r\n",
       "L 274.9254 108.986118 \r\n",
       "L 277.487384 108.830085 \r\n",
       "L 280.049367 108.251373 \r\n",
       "L 282.611351 107.171742 \r\n",
       "L 285.173334 105.521908 \r\n",
       "L 287.735318 103.243321 \r\n",
       "L 290.297301 100.289671 \r\n",
       "L 292.859285 96.628115 \r\n",
       "L 295.421268 92.24016 \r\n",
       "L 297.983252 87.12221 \r\n",
       "L 300.545235 81.285747 \r\n",
       "L 303.107218 74.75714 \r\n",
       "L 305.669202 67.5771 \r\n",
       "L 308.231185 59.799781 \r\n",
       "L 310.793169 51.491558 \r\n",
       "L 313.355152 42.729504 \r\n",
       "L 315.917136 33.599611 \r\n",
       "L 318.479119 24.194793 \r\n",
       "\" style=\"fill:none;stroke:#800080;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_3\">\r\n",
       "    <path d=\"M 52.160938 170.28 \r\n",
       "L 52.160938 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_4\">\r\n",
       "    <path d=\"M 331.160937 170.28 \r\n",
       "L 331.160937 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_5\">\r\n",
       "    <path d=\"M 52.160938 170.28 \r\n",
       "L 331.160937 170.28 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_6\">\r\n",
       "    <path d=\"M 52.160938 7.2 \r\n",
       "L 331.160937 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_7\">\r\n",
       "    <path d=\"M 136.196596 152.827247 \r\n",
       "Q 163.580445 157.186826 189.860164 161.370625 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "    <path d=\"M 186.224357 158.766609 \r\n",
       "L 189.860164 161.370625 \r\n",
       "L 185.595466 162.716861 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"text_15\">\r\n",
       "    <!-- empirical risk -->\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 56.203125 29.59375 \r\n",
       "L 56.203125 25.203125 \r\n",
       "L 14.890625 25.203125 \r\n",
       "Q 15.484375 15.921875 20.484375 11.0625 \r\n",
       "Q 25.484375 6.203125 34.421875 6.203125 \r\n",
       "Q 39.59375 6.203125 44.453125 7.46875 \r\n",
       "Q 49.3125 8.734375 54.109375 11.28125 \r\n",
       "L 54.109375 2.78125 \r\n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \r\n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \r\n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \r\n",
       "Q 5.515625 13.8125 5.515625 26.8125 \r\n",
       "Q 5.515625 40.234375 12.765625 48.109375 \r\n",
       "Q 20.015625 56 32.328125 56 \r\n",
       "Q 43.359375 56 49.78125 48.890625 \r\n",
       "Q 56.203125 41.796875 56.203125 29.59375 \r\n",
       "z\r\n",
       "M 47.21875 32.234375 \r\n",
       "Q 47.125 39.59375 43.09375 43.984375 \r\n",
       "Q 39.0625 48.390625 32.421875 48.390625 \r\n",
       "Q 24.90625 48.390625 20.390625 44.140625 \r\n",
       "Q 15.875 39.890625 15.1875 32.171875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-101\"/>\r\n",
       "     <path d=\"M 52 44.1875 \r\n",
       "Q 55.375 50.25 60.0625 53.125 \r\n",
       "Q 64.75 56 71.09375 56 \r\n",
       "Q 79.640625 56 84.28125 50.015625 \r\n",
       "Q 88.921875 44.046875 88.921875 33.015625 \r\n",
       "L 88.921875 0 \r\n",
       "L 79.890625 0 \r\n",
       "L 79.890625 32.71875 \r\n",
       "Q 79.890625 40.578125 77.09375 44.375 \r\n",
       "Q 74.3125 48.1875 68.609375 48.1875 \r\n",
       "Q 61.625 48.1875 57.5625 43.546875 \r\n",
       "Q 53.515625 38.921875 53.515625 30.90625 \r\n",
       "L 53.515625 0 \r\n",
       "L 44.484375 0 \r\n",
       "L 44.484375 32.71875 \r\n",
       "Q 44.484375 40.625 41.703125 44.40625 \r\n",
       "Q 38.921875 48.1875 33.109375 48.1875 \r\n",
       "Q 26.21875 48.1875 22.15625 43.53125 \r\n",
       "Q 18.109375 38.875 18.109375 30.90625 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 21.1875 51.21875 25.484375 53.609375 \r\n",
       "Q 29.78125 56 35.6875 56 \r\n",
       "Q 41.65625 56 45.828125 52.96875 \r\n",
       "Q 50 49.953125 52 44.1875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-109\"/>\r\n",
       "     <path d=\"M 18.109375 8.203125 \r\n",
       "L 18.109375 -20.796875 \r\n",
       "L 9.078125 -20.796875 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.390625 \r\n",
       "Q 20.953125 51.265625 25.265625 53.625 \r\n",
       "Q 29.59375 56 35.59375 56 \r\n",
       "Q 45.5625 56 51.78125 48.09375 \r\n",
       "Q 58.015625 40.1875 58.015625 27.296875 \r\n",
       "Q 58.015625 14.40625 51.78125 6.484375 \r\n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \r\n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \r\n",
       "Q 20.953125 3.328125 18.109375 8.203125 \r\n",
       "z\r\n",
       "M 48.6875 27.296875 \r\n",
       "Q 48.6875 37.203125 44.609375 42.84375 \r\n",
       "Q 40.53125 48.484375 33.40625 48.484375 \r\n",
       "Q 26.265625 48.484375 22.1875 42.84375 \r\n",
       "Q 18.109375 37.203125 18.109375 27.296875 \r\n",
       "Q 18.109375 17.390625 22.1875 11.75 \r\n",
       "Q 26.265625 6.109375 33.40625 6.109375 \r\n",
       "Q 40.53125 6.109375 44.609375 11.75 \r\n",
       "Q 48.6875 17.390625 48.6875 27.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-112\"/>\r\n",
       "     <path d=\"M 48.78125 52.59375 \r\n",
       "L 48.78125 44.1875 \r\n",
       "Q 44.96875 46.296875 41.140625 47.34375 \r\n",
       "Q 37.3125 48.390625 33.40625 48.390625 \r\n",
       "Q 24.65625 48.390625 19.8125 42.84375 \r\n",
       "Q 14.984375 37.3125 14.984375 27.296875 \r\n",
       "Q 14.984375 17.28125 19.8125 11.734375 \r\n",
       "Q 24.65625 6.203125 33.40625 6.203125 \r\n",
       "Q 37.3125 6.203125 41.140625 7.25 \r\n",
       "Q 44.96875 8.296875 48.78125 10.40625 \r\n",
       "L 48.78125 2.09375 \r\n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \r\n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \r\n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \r\n",
       "Q 5.515625 14.109375 5.515625 27.296875 \r\n",
       "Q 5.515625 40.671875 12.859375 48.328125 \r\n",
       "Q 20.21875 56 33.015625 56 \r\n",
       "Q 37.15625 56 41.109375 55.140625 \r\n",
       "Q 45.0625 54.296875 48.78125 52.59375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-99\"/>\r\n",
       "     <path d=\"M 34.28125 27.484375 \r\n",
       "Q 23.390625 27.484375 19.1875 25 \r\n",
       "Q 14.984375 22.515625 14.984375 16.5 \r\n",
       "Q 14.984375 11.71875 18.140625 8.90625 \r\n",
       "Q 21.296875 6.109375 26.703125 6.109375 \r\n",
       "Q 34.1875 6.109375 38.703125 11.40625 \r\n",
       "Q 43.21875 16.703125 43.21875 25.484375 \r\n",
       "L 43.21875 27.484375 \r\n",
       "z\r\n",
       "M 52.203125 31.203125 \r\n",
       "L 52.203125 0 \r\n",
       "L 43.21875 0 \r\n",
       "L 43.21875 8.296875 \r\n",
       "Q 40.140625 3.328125 35.546875 0.953125 \r\n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \r\n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \r\n",
       "Q 6 8.015625 6 15.921875 \r\n",
       "Q 6 25.140625 12.171875 29.828125 \r\n",
       "Q 18.359375 34.515625 30.609375 34.515625 \r\n",
       "L 43.21875 34.515625 \r\n",
       "L 43.21875 35.40625 \r\n",
       "Q 43.21875 41.609375 39.140625 45 \r\n",
       "Q 35.0625 48.390625 27.6875 48.390625 \r\n",
       "Q 23 48.390625 18.546875 47.265625 \r\n",
       "Q 14.109375 46.140625 10.015625 43.890625 \r\n",
       "L 10.015625 52.203125 \r\n",
       "Q 14.9375 54.109375 19.578125 55.046875 \r\n",
       "Q 24.21875 56 28.609375 56 \r\n",
       "Q 40.484375 56 46.34375 49.84375 \r\n",
       "Q 52.203125 43.703125 52.203125 31.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-97\"/>\r\n",
       "     <path d=\"M 9.421875 75.984375 \r\n",
       "L 18.40625 75.984375 \r\n",
       "L 18.40625 0 \r\n",
       "L 9.421875 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-108\"/>\r\n",
       "     <path id=\"DejaVuSans-32\"/>\r\n",
       "    </defs>\r\n",
       "    <g transform=\"translate(64.842756 149.59054)scale(0.1 -0.1)\">\r\n",
       "     <use xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "     <use x=\"61.523438\" xlink:href=\"#DejaVuSans-109\"/>\r\n",
       "     <use x=\"158.935547\" xlink:href=\"#DejaVuSans-112\"/>\r\n",
       "     <use x=\"222.412109\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"250.195312\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "     <use x=\"291.308594\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"319.091797\" xlink:href=\"#DejaVuSans-99\"/>\r\n",
       "     <use x=\"374.072266\" xlink:href=\"#DejaVuSans-97\"/>\r\n",
       "     <use x=\"435.351562\" xlink:href=\"#DejaVuSans-108\"/>\r\n",
       "     <use x=\"463.134766\" xlink:href=\"#DejaVuSans-32\"/>\r\n",
       "     <use x=\"494.921875\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "     <use x=\"536.035156\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"563.818359\" xlink:href=\"#DejaVuSans-115\"/>\r\n",
       "     <use x=\"615.917969\" xlink:href=\"#DejaVuSans-107\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_8\">\r\n",
       "    <path d=\"M 214.496325 82.038808 \r\n",
       "Q 216.462916 111.748006 218.355662 140.341611 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "    <path d=\"M 220.087094 136.218245 \r\n",
       "L 218.355662 140.341611 \r\n",
       "L 216.095829 136.482446 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"text_16\">\r\n",
       "    <!-- expected risk -->\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 18.3125 70.21875 \r\n",
       "L 18.3125 54.6875 \r\n",
       "L 36.8125 54.6875 \r\n",
       "L 36.8125 47.703125 \r\n",
       "L 18.3125 47.703125 \r\n",
       "L 18.3125 18.015625 \r\n",
       "Q 18.3125 11.328125 20.140625 9.421875 \r\n",
       "Q 21.96875 7.515625 27.59375 7.515625 \r\n",
       "L 36.8125 7.515625 \r\n",
       "L 36.8125 0 \r\n",
       "L 27.59375 0 \r\n",
       "Q 17.1875 0 13.234375 3.875 \r\n",
       "Q 9.28125 7.765625 9.28125 18.015625 \r\n",
       "L 9.28125 47.703125 \r\n",
       "L 2.6875 47.703125 \r\n",
       "L 2.6875 54.6875 \r\n",
       "L 9.28125 54.6875 \r\n",
       "L 9.28125 70.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-116\"/>\r\n",
       "     <path d=\"M 45.40625 46.390625 \r\n",
       "L 45.40625 75.984375 \r\n",
       "L 54.390625 75.984375 \r\n",
       "L 54.390625 0 \r\n",
       "L 45.40625 0 \r\n",
       "L 45.40625 8.203125 \r\n",
       "Q 42.578125 3.328125 38.25 0.953125 \r\n",
       "Q 33.9375 -1.421875 27.875 -1.421875 \r\n",
       "Q 17.96875 -1.421875 11.734375 6.484375 \r\n",
       "Q 5.515625 14.40625 5.515625 27.296875 \r\n",
       "Q 5.515625 40.1875 11.734375 48.09375 \r\n",
       "Q 17.96875 56 27.875 56 \r\n",
       "Q 33.9375 56 38.25 53.625 \r\n",
       "Q 42.578125 51.265625 45.40625 46.390625 \r\n",
       "z\r\n",
       "M 14.796875 27.296875 \r\n",
       "Q 14.796875 17.390625 18.875 11.75 \r\n",
       "Q 22.953125 6.109375 30.078125 6.109375 \r\n",
       "Q 37.203125 6.109375 41.296875 11.75 \r\n",
       "Q 45.40625 17.390625 45.40625 27.296875 \r\n",
       "Q 45.40625 37.203125 41.296875 42.84375 \r\n",
       "Q 37.203125 48.484375 30.078125 48.484375 \r\n",
       "Q 22.953125 48.484375 18.875 42.84375 \r\n",
       "Q 14.796875 37.203125 14.796875 27.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-100\"/>\r\n",
       "    </defs>\r\n",
       "    <g transform=\"translate(180.132012 75.966279)scale(0.1 -0.1)\">\r\n",
       "     <use xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "     <use x=\"61.507812\" xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "     <use x=\"120.6875\" xlink:href=\"#DejaVuSans-112\"/>\r\n",
       "     <use x=\"184.164062\" xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "     <use x=\"245.6875\" xlink:href=\"#DejaVuSans-99\"/>\r\n",
       "     <use x=\"300.667969\" xlink:href=\"#DejaVuSans-116\"/>\r\n",
       "     <use x=\"339.876953\" xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "     <use x=\"401.400391\" xlink:href=\"#DejaVuSans-100\"/>\r\n",
       "     <use x=\"464.876953\" xlink:href=\"#DejaVuSans-32\"/>\r\n",
       "     <use x=\"496.664062\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "     <use x=\"537.777344\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"565.560547\" xlink:href=\"#DejaVuSans-115\"/>\r\n",
       "     <use x=\"617.660156\" xlink:href=\"#DejaVuSans-107\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"legend_1\">\r\n",
       "    <g id=\"patch_9\">\r\n",
       "     <path d=\"M 241.107813 44.55625 \r\n",
       "L 324.160938 44.55625 \r\n",
       "Q 326.160938 44.55625 326.160938 42.55625 \r\n",
       "L 326.160938 14.2 \r\n",
       "Q 326.160938 12.2 324.160938 12.2 \r\n",
       "L 241.107813 12.2 \r\n",
       "Q 239.107813 12.2 239.107813 14.2 \r\n",
       "L 239.107813 42.55625 \r\n",
       "Q 239.107813 44.55625 241.107813 44.55625 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"line2d_15\">\r\n",
       "     <path d=\"M 243.107813 20.298438 \r\n",
       "L 263.107813 20.298438 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"line2d_16\"/>\r\n",
       "    <g id=\"text_17\">\r\n",
       "     <!-- train error -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 54.890625 33.015625 \r\n",
       "L 54.890625 0 \r\n",
       "L 45.90625 0 \r\n",
       "L 45.90625 32.71875 \r\n",
       "Q 45.90625 40.484375 42.875 44.328125 \r\n",
       "Q 39.84375 48.1875 33.796875 48.1875 \r\n",
       "Q 26.515625 48.1875 22.3125 43.546875 \r\n",
       "Q 18.109375 38.921875 18.109375 30.90625 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 21.34375 51.125 25.703125 53.5625 \r\n",
       "Q 30.078125 56 35.796875 56 \r\n",
       "Q 45.21875 56 50.046875 50.171875 \r\n",
       "Q 54.890625 44.34375 54.890625 33.015625 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-110\"/>\r\n",
       "      <path d=\"M 30.609375 48.390625 \r\n",
       "Q 23.390625 48.390625 19.1875 42.75 \r\n",
       "Q 14.984375 37.109375 14.984375 27.296875 \r\n",
       "Q 14.984375 17.484375 19.15625 11.84375 \r\n",
       "Q 23.34375 6.203125 30.609375 6.203125 \r\n",
       "Q 37.796875 6.203125 41.984375 11.859375 \r\n",
       "Q 46.1875 17.53125 46.1875 27.296875 \r\n",
       "Q 46.1875 37.015625 41.984375 42.703125 \r\n",
       "Q 37.796875 48.390625 30.609375 48.390625 \r\n",
       "z\r\n",
       "M 30.609375 56 \r\n",
       "Q 42.328125 56 49.015625 48.375 \r\n",
       "Q 55.71875 40.765625 55.71875 27.296875 \r\n",
       "Q 55.71875 13.875 49.015625 6.21875 \r\n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \r\n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \r\n",
       "Q 5.515625 13.875 5.515625 27.296875 \r\n",
       "Q 5.515625 40.765625 12.171875 48.375 \r\n",
       "Q 18.84375 56 30.609375 56 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-111\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(271.107813 23.798438)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-116\"/>\r\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "      <use x=\"80.322266\" xlink:href=\"#DejaVuSans-97\"/>\r\n",
       "      <use x=\"141.601562\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "      <use x=\"169.384766\" xlink:href=\"#DejaVuSans-110\"/>\r\n",
       "      <use x=\"232.763672\" xlink:href=\"#DejaVuSans-32\"/>\r\n",
       "      <use x=\"264.550781\" xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "      <use x=\"326.074219\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "      <use x=\"367.171875\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "      <use x=\"408.253906\" xlink:href=\"#DejaVuSans-111\"/>\r\n",
       "      <use x=\"469.435547\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"line2d_17\">\r\n",
       "     <path d=\"M 243.107813 34.976562 \r\n",
       "L 263.107813 34.976562 \r\n",
       "\" style=\"fill:none;stroke:#800080;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"line2d_18\"/>\r\n",
       "    <g id=\"text_18\">\r\n",
       "     <!-- test error -->\r\n",
       "     <g transform=\"translate(271.107813 38.476562)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-116\"/>\r\n",
       "      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "      <use x=\"100.732422\" xlink:href=\"#DejaVuSans-115\"/>\r\n",
       "      <use x=\"152.832031\" xlink:href=\"#DejaVuSans-116\"/>\r\n",
       "      <use x=\"192.041016\" xlink:href=\"#DejaVuSans-32\"/>\r\n",
       "      <use x=\"223.828125\" xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "      <use x=\"285.351562\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "      <use x=\"326.449219\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "      <use x=\"367.53125\" xlink:href=\"#DejaVuSans-111\"/>\r\n",
       "      <use x=\"428.712891\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       " </g>\r\n",
       " <defs>\r\n",
       "  <clipPath id=\"pd1d762be4b\">\r\n",
       "   <rect height=\"163.08\" width=\"279\" x=\"52.160938\" y=\"7.2\"/>\r\n",
       "  </clipPath>\r\n",
       " </defs>\r\n",
       "</svg>\r\n"
      ],
      "text/plain": [
       "<Figure size 360x216 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def f(x): return x * np.cos(np.pi * x)\n",
    "def g(x): return f(x) + 0.2 * np.cos(5 * np.pi * x)\n",
    "\n",
    "d2l.set_figsize((5, 3))\n",
    "x = np.arange(0.5, 1.5, 0.01)\n",
    "fig_f, = d2l.plt.plot(x, f(x),label=\"train error\")\n",
    "fig_g, = d2l.plt.plot(x, g(x),'--', c='purple', label=\"test error\")\n",
    "fig_f.axes.annotate('empirical risk', (1.0, -1.2), (0.5, -1.1),arrowprops=dict(arrowstyle='->'))\n",
    "fig_g.axes.annotate('expected risk', (1.1, -1.05), (0.95, -0.5),arrowprops=dict(arrowstyle='->'))\n",
    "d2l.plt.xlabel('x')\n",
    "d2l.plt.ylabel('risk')\n",
    "d2l.plt.legend(loc=\"upper right\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_02rlac8",
    "id": "A39C014782E6405D8DFAA44E0ACF27E1",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 优化在深度学习中的挑战\n",
    "1. 局部最小值\n",
    "2. 鞍点\n",
    "3. 梯度消失"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_mw91a1z",
    "id": "D82C65CB587149489A446D5AF38840AB",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 局部最小值\n",
    "深度学习模型的目标函数可能有若干局部最优值。当一个优化问题的数值解在局部最优解附近时，由于目标函数有关解的梯度接近或变成零，最终迭代求得的数值解可能只令目标函数局部最小化而非全局最小化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_hb74ebt",
    "id": "FF518F2C0A474A2189DB5253F394E385",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "\n",
    "$$\n",
    "f(x) = x\\cos \\pi x\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "graffitiCellId": "id_jjnll27",
    "id": "4965DB27A9A347E58616D03D993E961F",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\r\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\r\n",
       "<svg height=\"180.65625pt\" version=\"1.1\" viewBox=\"0 0 305.329712 180.65625\" width=\"305.329712pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n",
       " <defs>\r\n",
       "  <style type=\"text/css\">\r\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\r\n",
       "  </style>\r\n",
       " </defs>\r\n",
       " <g id=\"figure_1\">\r\n",
       "  <g id=\"patch_1\">\r\n",
       "   <path d=\"M 0 180.65625 \r\n",
       "L 305.329712 180.65625 \r\n",
       "L 305.329712 0 \r\n",
       "L 0 0 \r\n",
       "z\r\n",
       "\" style=\"fill:none;\"/>\r\n",
       "  </g>\r\n",
       "  <g id=\"axes_1\">\r\n",
       "   <g id=\"patch_2\">\r\n",
       "    <path d=\"M 42.620313 143.1 \r\n",
       "L 293.720313 143.1 \r\n",
       "L 293.720313 7.2 \r\n",
       "L 42.620313 7.2 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_1\">\r\n",
       "    <g id=\"xtick_1\">\r\n",
       "     <g id=\"line2d_1\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L 0 3.5 \r\n",
       "\" id=\"m36c5e0a435\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.033949\" xlink:href=\"#m36c5e0a435\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_1\">\r\n",
       "      <!-- −1.0 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.59375 35.5 \r\n",
       "L 73.1875 35.5 \r\n",
       "L 73.1875 27.203125 \r\n",
       "L 10.59375 27.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-8722\"/>\r\n",
       "       <path d=\"M 12.40625 8.296875 \r\n",
       "L 28.515625 8.296875 \r\n",
       "L 28.515625 63.921875 \r\n",
       "L 10.984375 60.40625 \r\n",
       "L 10.984375 69.390625 \r\n",
       "L 28.421875 72.90625 \r\n",
       "L 38.28125 72.90625 \r\n",
       "L 38.28125 8.296875 \r\n",
       "L 54.390625 8.296875 \r\n",
       "L 54.390625 0 \r\n",
       "L 12.40625 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-49\"/>\r\n",
       "       <path d=\"M 10.6875 12.40625 \r\n",
       "L 21 12.40625 \r\n",
       "L 21 0 \r\n",
       "L 10.6875 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-46\"/>\r\n",
       "       <path d=\"M 31.78125 66.40625 \r\n",
       "Q 24.171875 66.40625 20.328125 58.90625 \r\n",
       "Q 16.5 51.421875 16.5 36.375 \r\n",
       "Q 16.5 21.390625 20.328125 13.890625 \r\n",
       "Q 24.171875 6.390625 31.78125 6.390625 \r\n",
       "Q 39.453125 6.390625 43.28125 13.890625 \r\n",
       "Q 47.125 21.390625 47.125 36.375 \r\n",
       "Q 47.125 51.421875 43.28125 58.90625 \r\n",
       "Q 39.453125 66.40625 31.78125 66.40625 \r\n",
       "z\r\n",
       "M 31.78125 74.21875 \r\n",
       "Q 44.046875 74.21875 50.515625 64.515625 \r\n",
       "Q 56.984375 54.828125 56.984375 36.375 \r\n",
       "Q 56.984375 17.96875 50.515625 8.265625 \r\n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \r\n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \r\n",
       "Q 6.59375 17.96875 6.59375 36.375 \r\n",
       "Q 6.59375 54.828125 13.0625 64.515625 \r\n",
       "Q 19.53125 74.21875 31.78125 74.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-48\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(41.892543 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_2\">\r\n",
       "     <g id=\"line2d_2\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"93.391316\" xlink:href=\"#m36c5e0a435\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_2\">\r\n",
       "      <!-- −0.5 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.796875 72.90625 \r\n",
       "L 49.515625 72.90625 \r\n",
       "L 49.515625 64.59375 \r\n",
       "L 19.828125 64.59375 \r\n",
       "L 19.828125 46.734375 \r\n",
       "Q 21.96875 47.46875 24.109375 47.828125 \r\n",
       "Q 26.265625 48.1875 28.421875 48.1875 \r\n",
       "Q 40.625 48.1875 47.75 41.5 \r\n",
       "Q 54.890625 34.8125 54.890625 23.390625 \r\n",
       "Q 54.890625 11.625 47.5625 5.09375 \r\n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \r\n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \r\n",
       "Q 12.796875 0.140625 7.71875 1.703125 \r\n",
       "L 7.71875 11.625 \r\n",
       "Q 12.109375 9.234375 16.796875 8.0625 \r\n",
       "Q 21.484375 6.890625 26.703125 6.890625 \r\n",
       "Q 35.15625 6.890625 40.078125 11.328125 \r\n",
       "Q 45.015625 15.765625 45.015625 23.390625 \r\n",
       "Q 45.015625 31 40.078125 35.4375 \r\n",
       "Q 35.15625 39.890625 26.703125 39.890625 \r\n",
       "Q 22.75 39.890625 18.8125 39.015625 \r\n",
       "Q 14.890625 38.140625 10.796875 36.28125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-53\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(81.249909 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_3\">\r\n",
       "     <g id=\"line2d_3\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"132.748682\" xlink:href=\"#m36c5e0a435\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_3\">\r\n",
       "      <!-- 0.0 -->\r\n",
       "      <g transform=\"translate(124.79712 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_4\">\r\n",
       "     <g id=\"line2d_4\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"172.106049\" xlink:href=\"#m36c5e0a435\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_4\">\r\n",
       "      <!-- 0.5 -->\r\n",
       "      <g transform=\"translate(164.154487 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_5\">\r\n",
       "     <g id=\"line2d_5\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"211.463416\" xlink:href=\"#m36c5e0a435\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_5\">\r\n",
       "      <!-- 1.0 -->\r\n",
       "      <g transform=\"translate(203.511853 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_6\">\r\n",
       "     <g id=\"line2d_6\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"250.820783\" xlink:href=\"#m36c5e0a435\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_6\">\r\n",
       "      <!-- 1.5 -->\r\n",
       "      <g transform=\"translate(242.86922 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_7\">\r\n",
       "     <g id=\"line2d_7\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"290.178149\" xlink:href=\"#m36c5e0a435\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_7\">\r\n",
       "      <!-- 2.0 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 19.1875 8.296875 \r\n",
       "L 53.609375 8.296875 \r\n",
       "L 53.609375 0 \r\n",
       "L 7.328125 0 \r\n",
       "L 7.328125 8.296875 \r\n",
       "Q 12.9375 14.109375 22.625 23.890625 \r\n",
       "Q 32.328125 33.6875 34.8125 36.53125 \r\n",
       "Q 39.546875 41.84375 41.421875 45.53125 \r\n",
       "Q 43.3125 49.21875 43.3125 52.78125 \r\n",
       "Q 43.3125 58.59375 39.234375 62.25 \r\n",
       "Q 35.15625 65.921875 28.609375 65.921875 \r\n",
       "Q 23.96875 65.921875 18.8125 64.3125 \r\n",
       "Q 13.671875 62.703125 7.8125 59.421875 \r\n",
       "L 7.8125 69.390625 \r\n",
       "Q 13.765625 71.78125 18.9375 73 \r\n",
       "Q 24.125 74.21875 28.421875 74.21875 \r\n",
       "Q 39.75 74.21875 46.484375 68.546875 \r\n",
       "Q 53.21875 62.890625 53.21875 53.421875 \r\n",
       "Q 53.21875 48.921875 51.53125 44.890625 \r\n",
       "Q 49.859375 40.875 45.40625 35.40625 \r\n",
       "Q 44.1875 33.984375 37.640625 27.21875 \r\n",
       "Q 31.109375 20.453125 19.1875 8.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-50\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(282.226587 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_8\">\r\n",
       "     <!-- x -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 54.890625 54.6875 \r\n",
       "L 35.109375 28.078125 \r\n",
       "L 55.90625 0 \r\n",
       "L 45.3125 0 \r\n",
       "L 29.390625 21.484375 \r\n",
       "L 13.484375 0 \r\n",
       "L 2.875 0 \r\n",
       "L 24.125 28.609375 \r\n",
       "L 4.6875 54.6875 \r\n",
       "L 15.28125 54.6875 \r\n",
       "L 29.78125 35.203125 \r\n",
       "L 44.28125 54.6875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-120\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(165.210938 171.376563)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_2\">\r\n",
       "    <g id=\"ytick_1\">\r\n",
       "     <g id=\"line2d_8\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L -3.5 0 \r\n",
       "\" id=\"m7280d9fdaf\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m7280d9fdaf\" y=\"134.923853\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_9\">\r\n",
       "      <!-- −1 -->\r\n",
       "      <g transform=\"translate(20.878125 138.723072)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_2\">\r\n",
       "     <g id=\"line2d_9\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m7280d9fdaf\" y=\"91.622727\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_10\">\r\n",
       "      <!-- 0 -->\r\n",
       "      <g transform=\"translate(29.257813 95.421946)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_3\">\r\n",
       "     <g id=\"line2d_10\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#m7280d9fdaf\" y=\"48.321601\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_11\">\r\n",
       "      <!-- 1 -->\r\n",
       "      <g transform=\"translate(29.257813 52.12082)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_12\">\r\n",
       "     <!-- f(x) -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 37.109375 75.984375 \r\n",
       "L 37.109375 68.5 \r\n",
       "L 28.515625 68.5 \r\n",
       "Q 23.6875 68.5 21.796875 66.546875 \r\n",
       "Q 19.921875 64.59375 19.921875 59.515625 \r\n",
       "L 19.921875 54.6875 \r\n",
       "L 34.71875 54.6875 \r\n",
       "L 34.71875 47.703125 \r\n",
       "L 19.921875 47.703125 \r\n",
       "L 19.921875 0 \r\n",
       "L 10.890625 0 \r\n",
       "L 10.890625 47.703125 \r\n",
       "L 2.296875 47.703125 \r\n",
       "L 2.296875 54.6875 \r\n",
       "L 10.890625 54.6875 \r\n",
       "L 10.890625 58.5 \r\n",
       "Q 10.890625 67.625 15.140625 71.796875 \r\n",
       "Q 19.390625 75.984375 28.609375 75.984375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-102\"/>\r\n",
       "      <path d=\"M 31 75.875 \r\n",
       "Q 24.46875 64.65625 21.28125 53.65625 \r\n",
       "Q 18.109375 42.671875 18.109375 31.390625 \r\n",
       "Q 18.109375 20.125 21.3125 9.0625 \r\n",
       "Q 24.515625 -2 31 -13.1875 \r\n",
       "L 23.1875 -13.1875 \r\n",
       "Q 15.875 -1.703125 12.234375 9.375 \r\n",
       "Q 8.59375 20.453125 8.59375 31.390625 \r\n",
       "Q 8.59375 42.28125 12.203125 53.3125 \r\n",
       "Q 15.828125 64.359375 23.1875 75.875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-40\"/>\r\n",
       "      <path d=\"M 8.015625 75.875 \r\n",
       "L 15.828125 75.875 \r\n",
       "Q 23.140625 64.359375 26.78125 53.3125 \r\n",
       "Q 30.421875 42.28125 30.421875 31.390625 \r\n",
       "Q 30.421875 20.453125 26.78125 9.375 \r\n",
       "Q 23.140625 -1.703125 15.828125 -13.1875 \r\n",
       "L 8.015625 -13.1875 \r\n",
       "Q 14.5 -2 17.703125 9.0625 \r\n",
       "Q 20.90625 20.125 20.90625 31.390625 \r\n",
       "Q 20.90625 42.671875 17.703125 53.65625 \r\n",
       "Q 14.5 64.65625 8.015625 75.875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-41\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(14.798438 83.771094)rotate(-90)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-102\"/>\r\n",
       "      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-40\"/>\r\n",
       "      <use x=\"74.21875\" xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "      <use x=\"133.398438\" xlink:href=\"#DejaVuSans-41\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_11\">\r\n",
       "    <path clip-path=\"url(#pd74d7d3d01)\" d=\"M 54.033949 48.321601 \r\n",
       "L 61.905422 54.559091 \r\n",
       "L 69.776896 63.59765 \r\n",
       "L 77.648369 73.806493 \r\n",
       "L 85.519842 83.594257 \r\n",
       "L 93.391316 91.622727 \r\n",
       "L 101.262789 96.975041 \r\n",
       "L 109.134262 99.258256 \r\n",
       "L 117.005736 98.628997 \r\n",
       "L 124.877209 95.740909 \r\n",
       "L 132.748682 91.622727 \r\n",
       "L 140.620156 87.504545 \r\n",
       "L 148.491629 84.616458 \r\n",
       "L 156.363102 83.987198 \r\n",
       "L 164.234576 86.270414 \r\n",
       "L 172.106049 91.622727 \r\n",
       "L 179.977523 99.651198 \r\n",
       "L 187.848996 109.438962 \r\n",
       "L 195.720469 119.647805 \r\n",
       "L 203.591943 128.686364 \r\n",
       "L 211.463416 134.923853 \r\n",
       "L 219.334889 136.922727 \r\n",
       "L 227.206363 133.660344 \r\n",
       "L 235.077836 124.71002 \r\n",
       "L 242.949309 110.355825 \r\n",
       "L 250.820783 91.622727 \r\n",
       "L 258.692256 70.213473 \r\n",
       "L 266.563729 48.35473 \r\n",
       "L 274.435203 28.566303 \r\n",
       "L 282.306676 13.377273 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_3\">\r\n",
       "    <path d=\"M 42.620313 143.1 \r\n",
       "L 42.620313 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_4\">\r\n",
       "    <path d=\"M 293.720313 143.1 \r\n",
       "L 293.720313 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_5\">\r\n",
       "    <path d=\"M 42.620313 143.1 \r\n",
       "L 293.720313 143.1 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_6\">\r\n",
       "    <path d=\"M 42.620313 7.2 \r\n",
       "L 293.720313 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_7\">\r\n",
       "    <path d=\"M 109.113525 123.324204 \r\n",
       "Q 109.1229 113.886339 109.131165 105.566508 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "    <path d=\"M 107.127192 109.564519 \r\n",
       "L 109.131165 105.566508 \r\n",
       "L 111.12719 109.568492 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"text_13\">\r\n",
       "    <!-- local minimum -->\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 9.421875 75.984375 \r\n",
       "L 18.40625 75.984375 \r\n",
       "L 18.40625 0 \r\n",
       "L 9.421875 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-108\"/>\r\n",
       "     <path d=\"M 30.609375 48.390625 \r\n",
       "Q 23.390625 48.390625 19.1875 42.75 \r\n",
       "Q 14.984375 37.109375 14.984375 27.296875 \r\n",
       "Q 14.984375 17.484375 19.15625 11.84375 \r\n",
       "Q 23.34375 6.203125 30.609375 6.203125 \r\n",
       "Q 37.796875 6.203125 41.984375 11.859375 \r\n",
       "Q 46.1875 17.53125 46.1875 27.296875 \r\n",
       "Q 46.1875 37.015625 41.984375 42.703125 \r\n",
       "Q 37.796875 48.390625 30.609375 48.390625 \r\n",
       "z\r\n",
       "M 30.609375 56 \r\n",
       "Q 42.328125 56 49.015625 48.375 \r\n",
       "Q 55.71875 40.765625 55.71875 27.296875 \r\n",
       "Q 55.71875 13.875 49.015625 6.21875 \r\n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \r\n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \r\n",
       "Q 5.515625 13.875 5.515625 27.296875 \r\n",
       "Q 5.515625 40.765625 12.171875 48.375 \r\n",
       "Q 18.84375 56 30.609375 56 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-111\"/>\r\n",
       "     <path d=\"M 48.78125 52.59375 \r\n",
       "L 48.78125 44.1875 \r\n",
       "Q 44.96875 46.296875 41.140625 47.34375 \r\n",
       "Q 37.3125 48.390625 33.40625 48.390625 \r\n",
       "Q 24.65625 48.390625 19.8125 42.84375 \r\n",
       "Q 14.984375 37.3125 14.984375 27.296875 \r\n",
       "Q 14.984375 17.28125 19.8125 11.734375 \r\n",
       "Q 24.65625 6.203125 33.40625 6.203125 \r\n",
       "Q 37.3125 6.203125 41.140625 7.25 \r\n",
       "Q 44.96875 8.296875 48.78125 10.40625 \r\n",
       "L 48.78125 2.09375 \r\n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \r\n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \r\n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \r\n",
       "Q 5.515625 14.109375 5.515625 27.296875 \r\n",
       "Q 5.515625 40.671875 12.859375 48.328125 \r\n",
       "Q 20.21875 56 33.015625 56 \r\n",
       "Q 37.15625 56 41.109375 55.140625 \r\n",
       "Q 45.0625 54.296875 48.78125 52.59375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-99\"/>\r\n",
       "     <path d=\"M 34.28125 27.484375 \r\n",
       "Q 23.390625 27.484375 19.1875 25 \r\n",
       "Q 14.984375 22.515625 14.984375 16.5 \r\n",
       "Q 14.984375 11.71875 18.140625 8.90625 \r\n",
       "Q 21.296875 6.109375 26.703125 6.109375 \r\n",
       "Q 34.1875 6.109375 38.703125 11.40625 \r\n",
       "Q 43.21875 16.703125 43.21875 25.484375 \r\n",
       "L 43.21875 27.484375 \r\n",
       "z\r\n",
       "M 52.203125 31.203125 \r\n",
       "L 52.203125 0 \r\n",
       "L 43.21875 0 \r\n",
       "L 43.21875 8.296875 \r\n",
       "Q 40.140625 3.328125 35.546875 0.953125 \r\n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \r\n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \r\n",
       "Q 6 8.015625 6 15.921875 \r\n",
       "Q 6 25.140625 12.171875 29.828125 \r\n",
       "Q 18.359375 34.515625 30.609375 34.515625 \r\n",
       "L 43.21875 34.515625 \r\n",
       "L 43.21875 35.40625 \r\n",
       "Q 43.21875 41.609375 39.140625 45 \r\n",
       "Q 35.0625 48.390625 27.6875 48.390625 \r\n",
       "Q 23 48.390625 18.546875 47.265625 \r\n",
       "Q 14.109375 46.140625 10.015625 43.890625 \r\n",
       "L 10.015625 52.203125 \r\n",
       "Q 14.9375 54.109375 19.578125 55.046875 \r\n",
       "Q 24.21875 56 28.609375 56 \r\n",
       "Q 40.484375 56 46.34375 49.84375 \r\n",
       "Q 52.203125 43.703125 52.203125 31.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-97\"/>\r\n",
       "     <path id=\"DejaVuSans-32\"/>\r\n",
       "     <path d=\"M 52 44.1875 \r\n",
       "Q 55.375 50.25 60.0625 53.125 \r\n",
       "Q 64.75 56 71.09375 56 \r\n",
       "Q 79.640625 56 84.28125 50.015625 \r\n",
       "Q 88.921875 44.046875 88.921875 33.015625 \r\n",
       "L 88.921875 0 \r\n",
       "L 79.890625 0 \r\n",
       "L 79.890625 32.71875 \r\n",
       "Q 79.890625 40.578125 77.09375 44.375 \r\n",
       "Q 74.3125 48.1875 68.609375 48.1875 \r\n",
       "Q 61.625 48.1875 57.5625 43.546875 \r\n",
       "Q 53.515625 38.921875 53.515625 30.90625 \r\n",
       "L 53.515625 0 \r\n",
       "L 44.484375 0 \r\n",
       "L 44.484375 32.71875 \r\n",
       "Q 44.484375 40.625 41.703125 44.40625 \r\n",
       "Q 38.921875 48.1875 33.109375 48.1875 \r\n",
       "Q 26.21875 48.1875 22.15625 43.53125 \r\n",
       "Q 18.109375 38.875 18.109375 30.90625 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 21.1875 51.21875 25.484375 53.609375 \r\n",
       "Q 29.78125 56 35.6875 56 \r\n",
       "Q 41.65625 56 45.828125 52.96875 \r\n",
       "Q 50 49.953125 52 44.1875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-109\"/>\r\n",
       "     <path d=\"M 9.421875 54.6875 \r\n",
       "L 18.40625 54.6875 \r\n",
       "L 18.40625 0 \r\n",
       "L 9.421875 0 \r\n",
       "z\r\n",
       "M 9.421875 75.984375 \r\n",
       "L 18.40625 75.984375 \r\n",
       "L 18.40625 64.59375 \r\n",
       "L 9.421875 64.59375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-105\"/>\r\n",
       "     <path d=\"M 54.890625 33.015625 \r\n",
       "L 54.890625 0 \r\n",
       "L 45.90625 0 \r\n",
       "L 45.90625 32.71875 \r\n",
       "Q 45.90625 40.484375 42.875 44.328125 \r\n",
       "Q 39.84375 48.1875 33.796875 48.1875 \r\n",
       "Q 26.515625 48.1875 22.3125 43.546875 \r\n",
       "Q 18.109375 38.921875 18.109375 30.90625 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 21.34375 51.125 25.703125 53.5625 \r\n",
       "Q 30.078125 56 35.796875 56 \r\n",
       "Q 45.21875 56 50.046875 50.171875 \r\n",
       "Q 54.890625 44.34375 54.890625 33.015625 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-110\"/>\r\n",
       "     <path d=\"M 8.5 21.578125 \r\n",
       "L 8.5 54.6875 \r\n",
       "L 17.484375 54.6875 \r\n",
       "L 17.484375 21.921875 \r\n",
       "Q 17.484375 14.15625 20.5 10.265625 \r\n",
       "Q 23.53125 6.390625 29.59375 6.390625 \r\n",
       "Q 36.859375 6.390625 41.078125 11.03125 \r\n",
       "Q 45.3125 15.671875 45.3125 23.6875 \r\n",
       "L 45.3125 54.6875 \r\n",
       "L 54.296875 54.6875 \r\n",
       "L 54.296875 0 \r\n",
       "L 45.3125 0 \r\n",
       "L 45.3125 8.40625 \r\n",
       "Q 42.046875 3.421875 37.71875 1 \r\n",
       "Q 33.40625 -1.421875 27.6875 -1.421875 \r\n",
       "Q 18.265625 -1.421875 13.375 4.4375 \r\n",
       "Q 8.5 10.296875 8.5 21.578125 \r\n",
       "z\r\n",
       "M 31.109375 56 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-117\"/>\r\n",
       "    </defs>\r\n",
       "    <g transform=\"translate(72.138338 134.923853)scale(0.1 -0.1)\">\r\n",
       "     <use xlink:href=\"#DejaVuSans-108\"/>\r\n",
       "     <use x=\"27.783203\" xlink:href=\"#DejaVuSans-111\"/>\r\n",
       "     <use x=\"88.964844\" xlink:href=\"#DejaVuSans-99\"/>\r\n",
       "     <use x=\"143.945312\" xlink:href=\"#DejaVuSans-97\"/>\r\n",
       "     <use x=\"205.224609\" xlink:href=\"#DejaVuSans-108\"/>\r\n",
       "     <use x=\"233.007812\" xlink:href=\"#DejaVuSans-32\"/>\r\n",
       "     <use x=\"264.794922\" xlink:href=\"#DejaVuSans-109\"/>\r\n",
       "     <use x=\"362.207031\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"389.990234\" xlink:href=\"#DejaVuSans-110\"/>\r\n",
       "     <use x=\"453.369141\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"481.152344\" xlink:href=\"#DejaVuSans-109\"/>\r\n",
       "     <use x=\"578.564453\" xlink:href=\"#DejaVuSans-117\"/>\r\n",
       "     <use x=\"641.943359\" xlink:href=\"#DejaVuSans-109\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_8\">\r\n",
       "    <path d=\"M 220.407145 63.062567 \r\n",
       "Q 219.88643 96.908884 219.382913 129.637299 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "    <path d=\"M 221.444207 125.668538 \r\n",
       "L 219.382913 129.637299 \r\n",
       "L 217.444681 125.607006 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"text_14\">\r\n",
       "    <!-- global minimum -->\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 45.40625 27.984375 \r\n",
       "Q 45.40625 37.75 41.375 43.109375 \r\n",
       "Q 37.359375 48.484375 30.078125 48.484375 \r\n",
       "Q 22.859375 48.484375 18.828125 43.109375 \r\n",
       "Q 14.796875 37.75 14.796875 27.984375 \r\n",
       "Q 14.796875 18.265625 18.828125 12.890625 \r\n",
       "Q 22.859375 7.515625 30.078125 7.515625 \r\n",
       "Q 37.359375 7.515625 41.375 12.890625 \r\n",
       "Q 45.40625 18.265625 45.40625 27.984375 \r\n",
       "z\r\n",
       "M 54.390625 6.78125 \r\n",
       "Q 54.390625 -7.171875 48.1875 -13.984375 \r\n",
       "Q 42 -20.796875 29.203125 -20.796875 \r\n",
       "Q 24.46875 -20.796875 20.265625 -20.09375 \r\n",
       "Q 16.0625 -19.390625 12.109375 -17.921875 \r\n",
       "L 12.109375 -9.1875 \r\n",
       "Q 16.0625 -11.328125 19.921875 -12.34375 \r\n",
       "Q 23.78125 -13.375 27.78125 -13.375 \r\n",
       "Q 36.625 -13.375 41.015625 -8.765625 \r\n",
       "Q 45.40625 -4.15625 45.40625 5.171875 \r\n",
       "L 45.40625 9.625 \r\n",
       "Q 42.625 4.78125 38.28125 2.390625 \r\n",
       "Q 33.9375 0 27.875 0 \r\n",
       "Q 17.828125 0 11.671875 7.65625 \r\n",
       "Q 5.515625 15.328125 5.515625 27.984375 \r\n",
       "Q 5.515625 40.671875 11.671875 48.328125 \r\n",
       "Q 17.828125 56 27.875 56 \r\n",
       "Q 33.9375 56 38.28125 53.609375 \r\n",
       "Q 42.625 51.21875 45.40625 46.390625 \r\n",
       "L 45.40625 54.6875 \r\n",
       "L 54.390625 54.6875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-103\"/>\r\n",
       "     <path d=\"M 48.6875 27.296875 \r\n",
       "Q 48.6875 37.203125 44.609375 42.84375 \r\n",
       "Q 40.53125 48.484375 33.40625 48.484375 \r\n",
       "Q 26.265625 48.484375 22.1875 42.84375 \r\n",
       "Q 18.109375 37.203125 18.109375 27.296875 \r\n",
       "Q 18.109375 17.390625 22.1875 11.75 \r\n",
       "Q 26.265625 6.109375 33.40625 6.109375 \r\n",
       "Q 40.53125 6.109375 44.609375 11.75 \r\n",
       "Q 48.6875 17.390625 48.6875 27.296875 \r\n",
       "z\r\n",
       "M 18.109375 46.390625 \r\n",
       "Q 20.953125 51.265625 25.265625 53.625 \r\n",
       "Q 29.59375 56 35.59375 56 \r\n",
       "Q 45.5625 56 51.78125 48.09375 \r\n",
       "Q 58.015625 40.1875 58.015625 27.296875 \r\n",
       "Q 58.015625 14.40625 51.78125 6.484375 \r\n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \r\n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \r\n",
       "Q 20.953125 3.328125 18.109375 8.203125 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 75.984375 \r\n",
       "L 18.109375 75.984375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-98\"/>\r\n",
       "    </defs>\r\n",
       "    <g transform=\"translate(179.977523 56.981826)scale(0.1 -0.1)\">\r\n",
       "     <use xlink:href=\"#DejaVuSans-103\"/>\r\n",
       "     <use x=\"63.476562\" xlink:href=\"#DejaVuSans-108\"/>\r\n",
       "     <use x=\"91.259766\" xlink:href=\"#DejaVuSans-111\"/>\r\n",
       "     <use x=\"152.441406\" xlink:href=\"#DejaVuSans-98\"/>\r\n",
       "     <use x=\"215.917969\" xlink:href=\"#DejaVuSans-97\"/>\r\n",
       "     <use x=\"277.197266\" xlink:href=\"#DejaVuSans-108\"/>\r\n",
       "     <use x=\"304.980469\" xlink:href=\"#DejaVuSans-32\"/>\r\n",
       "     <use x=\"336.767578\" xlink:href=\"#DejaVuSans-109\"/>\r\n",
       "     <use x=\"434.179688\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"461.962891\" xlink:href=\"#DejaVuSans-110\"/>\r\n",
       "     <use x=\"525.341797\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"553.125\" xlink:href=\"#DejaVuSans-109\"/>\r\n",
       "     <use x=\"650.537109\" xlink:href=\"#DejaVuSans-117\"/>\r\n",
       "     <use x=\"713.916016\" xlink:href=\"#DejaVuSans-109\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       " </g>\r\n",
       " <defs>\r\n",
       "  <clipPath id=\"pd74d7d3d01\">\r\n",
       "   <rect height=\"135.9\" width=\"251.1\" x=\"42.620313\" y=\"7.2\"/>\r\n",
       "  </clipPath>\r\n",
       " </defs>\r\n",
       "</svg>\r\n"
      ],
      "text/plain": [
       "<Figure size 324x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def f(x):\n",
    "    return x * np.cos(np.pi * x)\n",
    "\n",
    "d2l.set_figsize((4.5, 2.5))\n",
    "x = np.arange(-1.0, 2.0, 0.1)\n",
    "fig,  = d2l.plt.plot(x, f(x))\n",
    "fig.axes.annotate('local minimum', xy=(-0.3, -0.25), xytext=(-0.77, -1.0),\n",
    "                  arrowprops=dict(arrowstyle='->'))\n",
    "fig.axes.annotate('global minimum', xy=(1.1, -0.95), xytext=(0.6, 0.8),\n",
    "                  arrowprops=dict(arrowstyle='->'))\n",
    "d2l.plt.xlabel('x')\n",
    "d2l.plt.ylabel('f(x)');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_veam6dy",
    "id": "8ABF7DD2A93444B18FDE4D95AE80F5A9",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 鞍点\n",
    "刚刚我们提到，梯度接近或变成零可能是由于当前解在局部最优解附近造成的。事实上，另一种可能性是当前解在鞍点（saddle point）附近\n",
    "假设一个函数的输入为 𝑘k 维向量，输出为标量，那么它的海森矩阵（Hessian matrix）有 𝑘k 个特征值。该函数在梯度为0的位置上可能是局部最小值、局部最大值或者鞍点。\n",
    "\n",
    "当函数的海森矩阵在梯度为零的位置上的特征值全为正时，该函数得到局部最小值。\n",
    "\n",
    "当函数的海森矩阵在梯度为零的位置上的特征值全为负时，该函数得到局部最大值。\n",
    "\n",
    "当函数的海森矩阵在梯度为零的位置上的特征值有正有负时，该函数得到鞍点。\n",
    "\n",
    "随机矩阵理论告诉我们，对于一个大的高斯随机矩阵来说，任一特征值是正或者是负的概率都是0.5 。那么，以上第一种情况的概率为$0.5^{k}$。由于深度学习模型参数通常都是高维的（ 𝑘 很大），目标函数的鞍点通常比局部最小值更常见。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "graffitiCellId": "id_8qeu2hn",
    "id": "02890A049EE14E1D91FD5198DEDA3FFD",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\r\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\r\n",
       "<svg height=\"180.65625pt\" version=\"1.1\" viewBox=\"0 0 300.920313 180.65625\" width=\"300.920313pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n",
       " <defs>\r\n",
       "  <style type=\"text/css\">\r\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\r\n",
       "  </style>\r\n",
       " </defs>\r\n",
       " <g id=\"figure_1\">\r\n",
       "  <g id=\"patch_1\">\r\n",
       "   <path d=\"M 0 180.65625 \r\n",
       "L 300.920313 180.65625 \r\n",
       "L 300.920313 0 \r\n",
       "L 0 0 \r\n",
       "z\r\n",
       "\" style=\"fill:none;\"/>\r\n",
       "  </g>\r\n",
       "  <g id=\"axes_1\">\r\n",
       "   <g id=\"patch_2\">\r\n",
       "    <path d=\"M 42.620313 143.1 \r\n",
       "L 293.720313 143.1 \r\n",
       "L 293.720313 7.2 \r\n",
       "L 42.620313 7.2 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_1\">\r\n",
       "    <g id=\"xtick_1\">\r\n",
       "     <g id=\"line2d_1\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L 0 3.5 \r\n",
       "\" id=\"m3c4fdc99de\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"54.033949\" xlink:href=\"#m3c4fdc99de\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_1\">\r\n",
       "      <!-- −2 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.59375 35.5 \r\n",
       "L 73.1875 35.5 \r\n",
       "L 73.1875 27.203125 \r\n",
       "L 10.59375 27.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-8722\"/>\r\n",
       "       <path d=\"M 19.1875 8.296875 \r\n",
       "L 53.609375 8.296875 \r\n",
       "L 53.609375 0 \r\n",
       "L 7.328125 0 \r\n",
       "L 7.328125 8.296875 \r\n",
       "Q 12.9375 14.109375 22.625 23.890625 \r\n",
       "Q 32.328125 33.6875 34.8125 36.53125 \r\n",
       "Q 39.546875 41.84375 41.421875 45.53125 \r\n",
       "Q 43.3125 49.21875 43.3125 52.78125 \r\n",
       "Q 43.3125 58.59375 39.234375 62.25 \r\n",
       "Q 35.15625 65.921875 28.609375 65.921875 \r\n",
       "Q 23.96875 65.921875 18.8125 64.3125 \r\n",
       "Q 13.671875 62.703125 7.8125 59.421875 \r\n",
       "L 7.8125 69.390625 \r\n",
       "Q 13.765625 71.78125 18.9375 73 \r\n",
       "Q 24.125 74.21875 28.421875 74.21875 \r\n",
       "Q 39.75 74.21875 46.484375 68.546875 \r\n",
       "Q 53.21875 62.890625 53.21875 53.421875 \r\n",
       "Q 53.21875 48.921875 51.53125 44.890625 \r\n",
       "Q 49.859375 40.875 45.40625 35.40625 \r\n",
       "Q 44.1875 33.984375 37.640625 27.21875 \r\n",
       "Q 31.109375 20.453125 19.1875 8.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-50\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(46.662855 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_2\">\r\n",
       "     <g id=\"line2d_2\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"112.565417\" xlink:href=\"#m3c4fdc99de\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_2\">\r\n",
       "      <!-- −1 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 12.40625 8.296875 \r\n",
       "L 28.515625 8.296875 \r\n",
       "L 28.515625 63.921875 \r\n",
       "L 10.984375 60.40625 \r\n",
       "L 10.984375 69.390625 \r\n",
       "L 28.421875 72.90625 \r\n",
       "L 38.28125 72.90625 \r\n",
       "L 38.28125 8.296875 \r\n",
       "L 54.390625 8.296875 \r\n",
       "L 54.390625 0 \r\n",
       "L 12.40625 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-49\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(105.194324 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_3\">\r\n",
       "     <g id=\"line2d_3\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"171.096886\" xlink:href=\"#m3c4fdc99de\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_3\">\r\n",
       "      <!-- 0 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 31.78125 66.40625 \r\n",
       "Q 24.171875 66.40625 20.328125 58.90625 \r\n",
       "Q 16.5 51.421875 16.5 36.375 \r\n",
       "Q 16.5 21.390625 20.328125 13.890625 \r\n",
       "Q 24.171875 6.390625 31.78125 6.390625 \r\n",
       "Q 39.453125 6.390625 43.28125 13.890625 \r\n",
       "Q 47.125 21.390625 47.125 36.375 \r\n",
       "Q 47.125 51.421875 43.28125 58.90625 \r\n",
       "Q 39.453125 66.40625 31.78125 66.40625 \r\n",
       "z\r\n",
       "M 31.78125 74.21875 \r\n",
       "Q 44.046875 74.21875 50.515625 64.515625 \r\n",
       "Q 56.984375 54.828125 56.984375 36.375 \r\n",
       "Q 56.984375 17.96875 50.515625 8.265625 \r\n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \r\n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \r\n",
       "Q 6.59375 17.96875 6.59375 36.375 \r\n",
       "Q 6.59375 54.828125 13.0625 64.515625 \r\n",
       "Q 19.53125 74.21875 31.78125 74.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-48\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(167.915636 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_4\">\r\n",
       "     <g id=\"line2d_4\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"229.628354\" xlink:href=\"#m3c4fdc99de\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_4\">\r\n",
       "      <!-- 1 -->\r\n",
       "      <g transform=\"translate(226.447104 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_5\">\r\n",
       "     <g id=\"line2d_5\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"288.159823\" xlink:href=\"#m3c4fdc99de\" y=\"143.1\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_5\">\r\n",
       "      <!-- 2 -->\r\n",
       "      <g transform=\"translate(284.978573 157.698438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_6\">\r\n",
       "     <!-- x -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 54.890625 54.6875 \r\n",
       "L 35.109375 28.078125 \r\n",
       "L 55.90625 0 \r\n",
       "L 45.3125 0 \r\n",
       "L 29.390625 21.484375 \r\n",
       "L 13.484375 0 \r\n",
       "L 2.875 0 \r\n",
       "L 24.125 28.609375 \r\n",
       "L 4.6875 54.6875 \r\n",
       "L 15.28125 54.6875 \r\n",
       "L 29.78125 35.203125 \r\n",
       "L 44.28125 54.6875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-120\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(165.210938 171.376563)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_2\">\r\n",
       "    <g id=\"ytick_1\">\r\n",
       "     <g id=\"line2d_6\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L -3.5 0 \r\n",
       "\" id=\"mbe78a832f6\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mbe78a832f6\" y=\"111.979167\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_7\">\r\n",
       "      <!-- −5 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.796875 72.90625 \r\n",
       "L 49.515625 72.90625 \r\n",
       "L 49.515625 64.59375 \r\n",
       "L 19.828125 64.59375 \r\n",
       "L 19.828125 46.734375 \r\n",
       "Q 21.96875 47.46875 24.109375 47.828125 \r\n",
       "Q 26.265625 48.1875 28.421875 48.1875 \r\n",
       "Q 40.625 48.1875 47.75 41.5 \r\n",
       "Q 54.890625 34.8125 54.890625 23.390625 \r\n",
       "Q 54.890625 11.625 47.5625 5.09375 \r\n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \r\n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \r\n",
       "Q 12.796875 0.140625 7.71875 1.703125 \r\n",
       "L 7.71875 11.625 \r\n",
       "Q 12.109375 9.234375 16.796875 8.0625 \r\n",
       "Q 21.484375 6.890625 26.703125 6.890625 \r\n",
       "Q 35.15625 6.890625 40.078125 11.328125 \r\n",
       "Q 45.015625 15.765625 45.015625 23.390625 \r\n",
       "Q 45.015625 31 40.078125 35.4375 \r\n",
       "Q 35.15625 39.890625 26.703125 39.890625 \r\n",
       "Q 22.75 39.890625 18.8125 39.015625 \r\n",
       "Q 14.890625 38.140625 10.796875 36.28125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-53\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(20.878125 115.778386)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_2\">\r\n",
       "     <g id=\"line2d_7\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mbe78a832f6\" y=\"70.406566\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_8\">\r\n",
       "      <!-- 0 -->\r\n",
       "      <g transform=\"translate(29.257813 74.205785)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_3\">\r\n",
       "     <g id=\"line2d_8\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"42.620313\" xlink:href=\"#mbe78a832f6\" y=\"28.833966\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_9\">\r\n",
       "      <!-- 5 -->\r\n",
       "      <g transform=\"translate(29.257813 32.633184)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_10\">\r\n",
       "     <!-- f(x) -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 37.109375 75.984375 \r\n",
       "L 37.109375 68.5 \r\n",
       "L 28.515625 68.5 \r\n",
       "Q 23.6875 68.5 21.796875 66.546875 \r\n",
       "Q 19.921875 64.59375 19.921875 59.515625 \r\n",
       "L 19.921875 54.6875 \r\n",
       "L 34.71875 54.6875 \r\n",
       "L 34.71875 47.703125 \r\n",
       "L 19.921875 47.703125 \r\n",
       "L 19.921875 0 \r\n",
       "L 10.890625 0 \r\n",
       "L 10.890625 47.703125 \r\n",
       "L 2.296875 47.703125 \r\n",
       "L 2.296875 54.6875 \r\n",
       "L 10.890625 54.6875 \r\n",
       "L 10.890625 58.5 \r\n",
       "Q 10.890625 67.625 15.140625 71.796875 \r\n",
       "Q 19.390625 75.984375 28.609375 75.984375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-102\"/>\r\n",
       "      <path d=\"M 31 75.875 \r\n",
       "Q 24.46875 64.65625 21.28125 53.65625 \r\n",
       "Q 18.109375 42.671875 18.109375 31.390625 \r\n",
       "Q 18.109375 20.125 21.3125 9.0625 \r\n",
       "Q 24.515625 -2 31 -13.1875 \r\n",
       "L 23.1875 -13.1875 \r\n",
       "Q 15.875 -1.703125 12.234375 9.375 \r\n",
       "Q 8.59375 20.453125 8.59375 31.390625 \r\n",
       "Q 8.59375 42.28125 12.203125 53.3125 \r\n",
       "Q 15.828125 64.359375 23.1875 75.875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-40\"/>\r\n",
       "      <path d=\"M 8.015625 75.875 \r\n",
       "L 15.828125 75.875 \r\n",
       "Q 23.140625 64.359375 26.78125 53.3125 \r\n",
       "Q 30.421875 42.28125 30.421875 31.390625 \r\n",
       "Q 30.421875 20.453125 26.78125 9.375 \r\n",
       "Q 23.140625 -1.703125 15.828125 -13.1875 \r\n",
       "L 8.015625 -13.1875 \r\n",
       "Q 14.5 -2 17.703125 9.0625 \r\n",
       "Q 20.90625 20.125 20.90625 31.390625 \r\n",
       "Q 20.90625 42.671875 17.703125 53.65625 \r\n",
       "Q 14.5 64.65625 8.015625 75.875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-41\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(14.798438 83.771094)rotate(-90)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-102\"/>\r\n",
       "      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-40\"/>\r\n",
       "      <use x=\"74.21875\" xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "      <use x=\"133.398438\" xlink:href=\"#DejaVuSans-41\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_9\">\r\n",
       "    <path clip-path=\"url(#pd59d6faedf)\" d=\"M 54.033949 136.922727 \r\n",
       "L 59.887096 127.43586 \r\n",
       "L 65.740243 118.896848 \r\n",
       "L 71.593389 111.255804 \r\n",
       "L 77.446536 104.462841 \r\n",
       "L 83.299683 98.468072 \r\n",
       "L 89.15283 93.221609 \r\n",
       "L 95.005977 88.673567 \r\n",
       "L 100.859124 84.774057 \r\n",
       "L 106.712271 81.473193 \r\n",
       "L 112.565417 78.721086 \r\n",
       "L 118.418564 76.467851 \r\n",
       "L 124.271711 74.663601 \r\n",
       "L 130.124858 73.258447 \r\n",
       "L 135.978005 72.202503 \r\n",
       "L 141.831152 71.445881 \r\n",
       "L 147.684299 70.938696 \r\n",
       "L 153.537445 70.631058 \r\n",
       "L 159.390592 70.473082 \r\n",
       "L 165.243739 70.414881 \r\n",
       "L 171.096886 70.406566 \r\n",
       "L 176.950033 70.398252 \r\n",
       "L 182.80318 70.34005 \r\n",
       "L 188.656326 70.182074 \r\n",
       "L 194.509473 69.874437 \r\n",
       "L 200.36262 69.367251 \r\n",
       "L 206.215767 68.61063 \r\n",
       "L 212.068914 67.554686 \r\n",
       "L 217.922061 66.149532 \r\n",
       "L 223.775208 64.345281 \r\n",
       "L 229.628354 62.092046 \r\n",
       "L 235.481501 59.33994 \r\n",
       "L 241.334648 56.039075 \r\n",
       "L 247.187795 52.139566 \r\n",
       "L 253.040942 47.591523 \r\n",
       "L 258.894089 42.345061 \r\n",
       "L 264.747236 36.350292 \r\n",
       "L 270.600382 29.557329 \r\n",
       "L 276.453529 21.916285 \r\n",
       "L 282.306676 13.377273 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_3\">\r\n",
       "    <path d=\"M 42.620313 143.1 \r\n",
       "L 42.620313 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_4\">\r\n",
       "    <path d=\"M 293.720313 143.1 \r\n",
       "L 293.720313 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_5\">\r\n",
       "    <path d=\"M 42.620313 143.1 \r\n",
       "L 293.720313 143.1 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_6\">\r\n",
       "    <path d=\"M 42.620313 7.2 \r\n",
       "L 293.720313 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_7\">\r\n",
       "    <path d=\"M 171.391806 100.37507 \r\n",
       "Q 171.254768 87.222572 171.129379 75.188047 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "    <path d=\"M 169.171162 79.208667 \r\n",
       "L 171.129379 75.188047 \r\n",
       "L 173.170944 79.166993 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"text_11\">\r\n",
       "    <!-- saddle point -->\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 44.28125 53.078125 \r\n",
       "L 44.28125 44.578125 \r\n",
       "Q 40.484375 46.53125 36.375 47.5 \r\n",
       "Q 32.28125 48.484375 27.875 48.484375 \r\n",
       "Q 21.1875 48.484375 17.84375 46.4375 \r\n",
       "Q 14.5 44.390625 14.5 40.28125 \r\n",
       "Q 14.5 37.15625 16.890625 35.375 \r\n",
       "Q 19.28125 33.59375 26.515625 31.984375 \r\n",
       "L 29.59375 31.296875 \r\n",
       "Q 39.15625 29.25 43.1875 25.515625 \r\n",
       "Q 47.21875 21.78125 47.21875 15.09375 \r\n",
       "Q 47.21875 7.46875 41.1875 3.015625 \r\n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \r\n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \r\n",
       "Q 10.6875 0.296875 5.421875 2 \r\n",
       "L 5.421875 11.28125 \r\n",
       "Q 10.40625 8.6875 15.234375 7.390625 \r\n",
       "Q 20.0625 6.109375 24.8125 6.109375 \r\n",
       "Q 31.15625 6.109375 34.5625 8.28125 \r\n",
       "Q 37.984375 10.453125 37.984375 14.40625 \r\n",
       "Q 37.984375 18.0625 35.515625 20.015625 \r\n",
       "Q 33.0625 21.96875 24.703125 23.78125 \r\n",
       "L 21.578125 24.515625 \r\n",
       "Q 13.234375 26.265625 9.515625 29.90625 \r\n",
       "Q 5.8125 33.546875 5.8125 39.890625 \r\n",
       "Q 5.8125 47.609375 11.28125 51.796875 \r\n",
       "Q 16.75 56 26.8125 56 \r\n",
       "Q 31.78125 56 36.171875 55.265625 \r\n",
       "Q 40.578125 54.546875 44.28125 53.078125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-115\"/>\r\n",
       "     <path d=\"M 34.28125 27.484375 \r\n",
       "Q 23.390625 27.484375 19.1875 25 \r\n",
       "Q 14.984375 22.515625 14.984375 16.5 \r\n",
       "Q 14.984375 11.71875 18.140625 8.90625 \r\n",
       "Q 21.296875 6.109375 26.703125 6.109375 \r\n",
       "Q 34.1875 6.109375 38.703125 11.40625 \r\n",
       "Q 43.21875 16.703125 43.21875 25.484375 \r\n",
       "L 43.21875 27.484375 \r\n",
       "z\r\n",
       "M 52.203125 31.203125 \r\n",
       "L 52.203125 0 \r\n",
       "L 43.21875 0 \r\n",
       "L 43.21875 8.296875 \r\n",
       "Q 40.140625 3.328125 35.546875 0.953125 \r\n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \r\n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \r\n",
       "Q 6 8.015625 6 15.921875 \r\n",
       "Q 6 25.140625 12.171875 29.828125 \r\n",
       "Q 18.359375 34.515625 30.609375 34.515625 \r\n",
       "L 43.21875 34.515625 \r\n",
       "L 43.21875 35.40625 \r\n",
       "Q 43.21875 41.609375 39.140625 45 \r\n",
       "Q 35.0625 48.390625 27.6875 48.390625 \r\n",
       "Q 23 48.390625 18.546875 47.265625 \r\n",
       "Q 14.109375 46.140625 10.015625 43.890625 \r\n",
       "L 10.015625 52.203125 \r\n",
       "Q 14.9375 54.109375 19.578125 55.046875 \r\n",
       "Q 24.21875 56 28.609375 56 \r\n",
       "Q 40.484375 56 46.34375 49.84375 \r\n",
       "Q 52.203125 43.703125 52.203125 31.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-97\"/>\r\n",
       "     <path d=\"M 45.40625 46.390625 \r\n",
       "L 45.40625 75.984375 \r\n",
       "L 54.390625 75.984375 \r\n",
       "L 54.390625 0 \r\n",
       "L 45.40625 0 \r\n",
       "L 45.40625 8.203125 \r\n",
       "Q 42.578125 3.328125 38.25 0.953125 \r\n",
       "Q 33.9375 -1.421875 27.875 -1.421875 \r\n",
       "Q 17.96875 -1.421875 11.734375 6.484375 \r\n",
       "Q 5.515625 14.40625 5.515625 27.296875 \r\n",
       "Q 5.515625 40.1875 11.734375 48.09375 \r\n",
       "Q 17.96875 56 27.875 56 \r\n",
       "Q 33.9375 56 38.25 53.625 \r\n",
       "Q 42.578125 51.265625 45.40625 46.390625 \r\n",
       "z\r\n",
       "M 14.796875 27.296875 \r\n",
       "Q 14.796875 17.390625 18.875 11.75 \r\n",
       "Q 22.953125 6.109375 30.078125 6.109375 \r\n",
       "Q 37.203125 6.109375 41.296875 11.75 \r\n",
       "Q 45.40625 17.390625 45.40625 27.296875 \r\n",
       "Q 45.40625 37.203125 41.296875 42.84375 \r\n",
       "Q 37.203125 48.484375 30.078125 48.484375 \r\n",
       "Q 22.953125 48.484375 18.875 42.84375 \r\n",
       "Q 14.796875 37.203125 14.796875 27.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-100\"/>\r\n",
       "     <path d=\"M 9.421875 75.984375 \r\n",
       "L 18.40625 75.984375 \r\n",
       "L 18.40625 0 \r\n",
       "L 9.421875 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-108\"/>\r\n",
       "     <path d=\"M 56.203125 29.59375 \r\n",
       "L 56.203125 25.203125 \r\n",
       "L 14.890625 25.203125 \r\n",
       "Q 15.484375 15.921875 20.484375 11.0625 \r\n",
       "Q 25.484375 6.203125 34.421875 6.203125 \r\n",
       "Q 39.59375 6.203125 44.453125 7.46875 \r\n",
       "Q 49.3125 8.734375 54.109375 11.28125 \r\n",
       "L 54.109375 2.78125 \r\n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \r\n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \r\n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \r\n",
       "Q 5.515625 13.8125 5.515625 26.8125 \r\n",
       "Q 5.515625 40.234375 12.765625 48.109375 \r\n",
       "Q 20.015625 56 32.328125 56 \r\n",
       "Q 43.359375 56 49.78125 48.890625 \r\n",
       "Q 56.203125 41.796875 56.203125 29.59375 \r\n",
       "z\r\n",
       "M 47.21875 32.234375 \r\n",
       "Q 47.125 39.59375 43.09375 43.984375 \r\n",
       "Q 39.0625 48.390625 32.421875 48.390625 \r\n",
       "Q 24.90625 48.390625 20.390625 44.140625 \r\n",
       "Q 15.875 39.890625 15.1875 32.171875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-101\"/>\r\n",
       "     <path id=\"DejaVuSans-32\"/>\r\n",
       "     <path d=\"M 18.109375 8.203125 \r\n",
       "L 18.109375 -20.796875 \r\n",
       "L 9.078125 -20.796875 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.390625 \r\n",
       "Q 20.953125 51.265625 25.265625 53.625 \r\n",
       "Q 29.59375 56 35.59375 56 \r\n",
       "Q 45.5625 56 51.78125 48.09375 \r\n",
       "Q 58.015625 40.1875 58.015625 27.296875 \r\n",
       "Q 58.015625 14.40625 51.78125 6.484375 \r\n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \r\n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \r\n",
       "Q 20.953125 3.328125 18.109375 8.203125 \r\n",
       "z\r\n",
       "M 48.6875 27.296875 \r\n",
       "Q 48.6875 37.203125 44.609375 42.84375 \r\n",
       "Q 40.53125 48.484375 33.40625 48.484375 \r\n",
       "Q 26.265625 48.484375 22.1875 42.84375 \r\n",
       "Q 18.109375 37.203125 18.109375 27.296875 \r\n",
       "Q 18.109375 17.390625 22.1875 11.75 \r\n",
       "Q 26.265625 6.109375 33.40625 6.109375 \r\n",
       "Q 40.53125 6.109375 44.609375 11.75 \r\n",
       "Q 48.6875 17.390625 48.6875 27.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-112\"/>\r\n",
       "     <path d=\"M 30.609375 48.390625 \r\n",
       "Q 23.390625 48.390625 19.1875 42.75 \r\n",
       "Q 14.984375 37.109375 14.984375 27.296875 \r\n",
       "Q 14.984375 17.484375 19.15625 11.84375 \r\n",
       "Q 23.34375 6.203125 30.609375 6.203125 \r\n",
       "Q 37.796875 6.203125 41.984375 11.859375 \r\n",
       "Q 46.1875 17.53125 46.1875 27.296875 \r\n",
       "Q 46.1875 37.015625 41.984375 42.703125 \r\n",
       "Q 37.796875 48.390625 30.609375 48.390625 \r\n",
       "z\r\n",
       "M 30.609375 56 \r\n",
       "Q 42.328125 56 49.015625 48.375 \r\n",
       "Q 55.71875 40.765625 55.71875 27.296875 \r\n",
       "Q 55.71875 13.875 49.015625 6.21875 \r\n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \r\n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \r\n",
       "Q 5.515625 13.875 5.515625 27.296875 \r\n",
       "Q 5.515625 40.765625 12.171875 48.375 \r\n",
       "Q 18.84375 56 30.609375 56 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-111\"/>\r\n",
       "     <path d=\"M 9.421875 54.6875 \r\n",
       "L 18.40625 54.6875 \r\n",
       "L 18.40625 0 \r\n",
       "L 9.421875 0 \r\n",
       "z\r\n",
       "M 9.421875 75.984375 \r\n",
       "L 18.40625 75.984375 \r\n",
       "L 18.40625 64.59375 \r\n",
       "L 9.421875 64.59375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-105\"/>\r\n",
       "     <path d=\"M 54.890625 33.015625 \r\n",
       "L 54.890625 0 \r\n",
       "L 45.90625 0 \r\n",
       "L 45.90625 32.71875 \r\n",
       "Q 45.90625 40.484375 42.875 44.328125 \r\n",
       "Q 39.84375 48.1875 33.796875 48.1875 \r\n",
       "Q 26.515625 48.1875 22.3125 43.546875 \r\n",
       "Q 18.109375 38.921875 18.109375 30.90625 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 21.34375 51.125 25.703125 53.5625 \r\n",
       "Q 30.078125 56 35.796875 56 \r\n",
       "Q 45.21875 56 50.046875 50.171875 \r\n",
       "Q 54.890625 44.34375 54.890625 33.015625 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-110\"/>\r\n",
       "     <path d=\"M 18.3125 70.21875 \r\n",
       "L 18.3125 54.6875 \r\n",
       "L 36.8125 54.6875 \r\n",
       "L 36.8125 47.703125 \r\n",
       "L 18.3125 47.703125 \r\n",
       "L 18.3125 18.015625 \r\n",
       "Q 18.3125 11.328125 20.140625 9.421875 \r\n",
       "Q 21.96875 7.515625 27.59375 7.515625 \r\n",
       "L 36.8125 7.515625 \r\n",
       "L 36.8125 0 \r\n",
       "L 27.59375 0 \r\n",
       "Q 17.1875 0 13.234375 3.875 \r\n",
       "Q 9.28125 7.765625 9.28125 18.015625 \r\n",
       "L 9.28125 47.703125 \r\n",
       "L 2.6875 47.703125 \r\n",
       "L 2.6875 54.6875 \r\n",
       "L 9.28125 54.6875 \r\n",
       "L 9.28125 70.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-116\"/>\r\n",
       "    </defs>\r\n",
       "    <g transform=\"translate(140.660522 111.979167)scale(0.1 -0.1)\">\r\n",
       "     <use xlink:href=\"#DejaVuSans-115\"/>\r\n",
       "     <use x=\"52.099609\" xlink:href=\"#DejaVuSans-97\"/>\r\n",
       "     <use x=\"113.378906\" xlink:href=\"#DejaVuSans-100\"/>\r\n",
       "     <use x=\"176.855469\" xlink:href=\"#DejaVuSans-100\"/>\r\n",
       "     <use x=\"240.332031\" xlink:href=\"#DejaVuSans-108\"/>\r\n",
       "     <use x=\"268.115234\" xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "     <use x=\"329.638672\" xlink:href=\"#DejaVuSans-32\"/>\r\n",
       "     <use x=\"361.425781\" xlink:href=\"#DejaVuSans-112\"/>\r\n",
       "     <use x=\"424.902344\" xlink:href=\"#DejaVuSans-111\"/>\r\n",
       "     <use x=\"486.083984\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"513.867188\" xlink:href=\"#DejaVuSans-110\"/>\r\n",
       "     <use x=\"577.246094\" xlink:href=\"#DejaVuSans-116\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       " </g>\r\n",
       " <defs>\r\n",
       "  <clipPath id=\"pd59d6faedf\">\r\n",
       "   <rect height=\"135.9\" width=\"251.1\" x=\"42.620313\" y=\"7.2\"/>\r\n",
       "  </clipPath>\r\n",
       " </defs>\r\n",
       "</svg>\r\n"
      ],
      "text/plain": [
       "<Figure size 324x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.arange(-2.0, 2.0, 0.1)\n",
    "fig, = d2l.plt.plot(x, x**3)\n",
    "fig.axes.annotate('saddle point', xy=(0, -0.2), xytext=(-0.52, -5.0),\n",
    "                  arrowprops=dict(arrowstyle='->'))\n",
    "d2l.plt.xlabel('x')\n",
    "d2l.plt.ylabel('f(x)');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_ffafz4q",
    "id": "3EA3A4DFAF1E458C8708218ADAB5794B",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "\n",
    "$$\n",
    "A=\\left[\\begin{array}{cccc}{\\frac{\\partial^{2} f}{\\partial x_{1}^{2}}} & {\\frac{\\partial^{2} f}{\\partial x_{1} \\partial x_{2}}} & {\\cdots} & {\\frac{\\partial^{2} f}{\\partial x_{1} \\partial x_{n}}} \\\\ {\\frac{\\partial^{2} f}{\\partial x_{2} \\partial x_{1}}} & {\\frac{\\partial^{2} f}{\\partial x_{2}^{2}}} & {\\cdots} & {\\frac{\\partial^{2} f}{\\partial x_{2} \\partial x_{n}}} \\\\ {\\vdots} & {\\vdots} & {\\ddots} & {\\vdots} \\\\ {\\frac{\\partial^{2} f}{\\partial x_{n} \\partial x_{1}}} & {\\frac{\\partial^{2} f}{\\partial x_{n} \\partial x_{2}}} & {\\cdots} & {\\frac{\\partial^{2} f}{\\partial x_{n}^{2}}}\\end{array}\\right]\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_tjexpt3",
    "id": "146D161023534174810D96162B8C8915",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "e.g."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "graffitiCellId": "id_n72qibc",
    "id": "974B5BA9119844BD95F3E6DE8FECFE15",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\r\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\r\n",
       "<svg height=\"231.84pt\" version=\"1.1\" viewBox=\"0 0 349.2 231.84\" width=\"349.2pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n",
       " <defs>\r\n",
       "  <style type=\"text/css\">\r\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\r\n",
       "  </style>\r\n",
       " </defs>\r\n",
       " <g id=\"figure_1\">\r\n",
       "  <g id=\"patch_1\">\r\n",
       "   <path d=\"M 0 231.84 \r\n",
       "L 349.2 231.84 \r\n",
       "L 349.2 0 \r\n",
       "L 0 0 \r\n",
       "z\r\n",
       "\" style=\"fill:none;\"/>\r\n",
       "  </g>\r\n",
       "  <g id=\"patch_2\">\r\n",
       "   <path d=\"M 7.2 224.64 \r\n",
       "L 342 224.64 \r\n",
       "L 342 7.2 \r\n",
       "L 7.2 7.2 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;\"/>\r\n",
       "  </g>\r\n",
       "  <g id=\"pane3d_1\">\r\n",
       "   <g id=\"patch_3\">\r\n",
       "    <path d=\"M 51.582914 176.593423 \r\n",
       "L 147.404425 123.282062 \r\n",
       "L 145.809099 21.412801 \r\n",
       "L 44.64491 69.311962 \r\n",
       "\" style=\"fill:#f2f2f2;opacity:0.5;stroke:#f2f2f2;stroke-linejoin:miter;\"/>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       "  <g id=\"pane3d_2\">\r\n",
       "   <g id=\"patch_4\">\r\n",
       "    <path d=\"M 147.404425 123.282062 \r\n",
       "L 302.626265 153.086553 \r\n",
       "L 309.120561 48.145466 \r\n",
       "L 145.809099 21.412801 \r\n",
       "\" style=\"fill:#e6e6e6;opacity:0.5;stroke:#e6e6e6;stroke-linejoin:miter;\"/>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       "  <g id=\"pane3d_3\">\r\n",
       "   <g id=\"patch_5\">\r\n",
       "    <path d=\"M 51.582914 176.593423 \r\n",
       "L 214.897263 211.305609 \r\n",
       "L 302.626265 153.086553 \r\n",
       "L 147.404425 123.282062 \r\n",
       "\" style=\"fill:#ececec;opacity:0.5;stroke:#ececec;stroke-linejoin:miter;\"/>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       "  <g id=\"axis3d_1\">\r\n",
       "   <g id=\"line2d_1\">\r\n",
       "    <path d=\"M 51.582914 176.593423 \r\n",
       "L 214.897263 211.305609 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"text_1\">\r\n",
       "    <!-- x -->\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 54.890625 54.6875 \r\n",
       "L 35.109375 28.078125 \r\n",
       "L 55.90625 0 \r\n",
       "L 45.3125 0 \r\n",
       "L 29.390625 21.484375 \r\n",
       "L 13.484375 0 \r\n",
       "L 2.875 0 \r\n",
       "L 24.125 28.609375 \r\n",
       "L 4.6875 54.6875 \r\n",
       "L 15.28125 54.6875 \r\n",
       "L 29.78125 35.203125 \r\n",
       "L 44.28125 54.6875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-120\"/>\r\n",
       "    </defs>\r\n",
       "    <g transform=\"translate(111.516779 224.825263)scale(0.1 -0.1)\">\r\n",
       "     <use xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"Line3DCollection_1\">\r\n",
       "    <path d=\"M 61.539611 178.709702 \r\n",
       "L 156.897598 125.104867 \r\n",
       "L 155.776615 23.044396 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "    <path d=\"M 131.3742 193.552925 \r\n",
       "L 223.371995 137.868764 \r\n",
       "L 225.646939 34.48156 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "    <path d=\"M 204.050753 209.000202 \r\n",
       "L 292.34945 151.113279 \r\n",
       "L 298.286139 46.371965 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_1\">\r\n",
       "    <g id=\"line2d_2\">\r\n",
       "     <path d=\"M 62.361169 178.247869 \r\n",
       "L 59.893431 179.635091 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_2\">\r\n",
       "     <!-- −1 -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 10.59375 35.5 \r\n",
       "L 73.1875 35.5 \r\n",
       "L 73.1875 27.203125 \r\n",
       "L 10.59375 27.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-8722\"/>\r\n",
       "      <path d=\"M 12.40625 8.296875 \r\n",
       "L 28.515625 8.296875 \r\n",
       "L 28.515625 63.921875 \r\n",
       "L 10.984375 60.40625 \r\n",
       "L 10.984375 69.390625 \r\n",
       "L 28.421875 72.90625 \r\n",
       "L 38.28125 72.90625 \r\n",
       "L 38.28125 8.296875 \r\n",
       "L 54.390625 8.296875 \r\n",
       "L 54.390625 0 \r\n",
       "L 12.40625 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-49\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(46.421865 199.10444)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "      <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_2\">\r\n",
       "    <g id=\"line2d_3\">\r\n",
       "     <path d=\"M 132.167936 193.072494 \r\n",
       "L 129.783706 194.515614 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_3\">\r\n",
       "     <!-- 0 -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 31.78125 66.40625 \r\n",
       "Q 24.171875 66.40625 20.328125 58.90625 \r\n",
       "Q 16.5 51.421875 16.5 36.375 \r\n",
       "Q 16.5 21.390625 20.328125 13.890625 \r\n",
       "Q 24.171875 6.390625 31.78125 6.390625 \r\n",
       "Q 39.453125 6.390625 43.28125 13.890625 \r\n",
       "Q 47.125 21.390625 47.125 36.375 \r\n",
       "Q 47.125 51.421875 43.28125 58.90625 \r\n",
       "Q 39.453125 66.40625 31.78125 66.40625 \r\n",
       "z\r\n",
       "M 31.78125 74.21875 \r\n",
       "Q 44.046875 74.21875 50.515625 64.515625 \r\n",
       "Q 56.984375 54.828125 56.984375 36.375 \r\n",
       "Q 56.984375 17.96875 50.515625 8.265625 \r\n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \r\n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \r\n",
       "Q 6.59375 17.96875 6.59375 36.375 \r\n",
       "Q 6.59375 54.828125 13.0625 64.515625 \r\n",
       "Q 19.53125 74.21875 31.78125 74.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-48\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(120.439142 214.23477)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_3\">\r\n",
       "    <g id=\"line2d_4\">\r\n",
       "     <path d=\"M 204.813702 208.500027 \r\n",
       "L 202.521894 210.002493 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_4\">\r\n",
       "     <!-- 1 -->\r\n",
       "     <g transform=\"translate(193.11438 229.982168)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       "  <g id=\"axis3d_2\">\r\n",
       "   <g id=\"line2d_5\">\r\n",
       "    <path d=\"M 302.626265 153.086553 \r\n",
       "L 214.897263 211.305609 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"text_5\">\r\n",
       "    <!-- y -->\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 32.171875 -5.078125 \r\n",
       "Q 28.375 -14.84375 24.75 -17.8125 \r\n",
       "Q 21.140625 -20.796875 15.09375 -20.796875 \r\n",
       "L 7.90625 -20.796875 \r\n",
       "L 7.90625 -13.28125 \r\n",
       "L 13.1875 -13.28125 \r\n",
       "Q 16.890625 -13.28125 18.9375 -11.515625 \r\n",
       "Q 21 -9.765625 23.484375 -3.21875 \r\n",
       "L 25.09375 0.875 \r\n",
       "L 2.984375 54.6875 \r\n",
       "L 12.5 54.6875 \r\n",
       "L 29.59375 11.921875 \r\n",
       "L 46.6875 54.6875 \r\n",
       "L 56.203125 54.6875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-121\"/>\r\n",
       "    </defs>\r\n",
       "    <g transform=\"translate(285.659129 207.381712)scale(0.1 -0.1)\">\r\n",
       "     <use xlink:href=\"#DejaVuSans-121\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"Line3DCollection_2\">\r\n",
       "    <path d=\"M 51.571961 66.032146 \r\n",
       "L 58.120339 172.956254 \r\n",
       "L 220.901308 207.321182 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "    <path d=\"M 97.209785 44.423576 \r\n",
       "L 101.278453 148.944758 \r\n",
       "L 260.469264 181.062944 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "    <path d=\"M 139.828335 24.244569 \r\n",
       "L 141.718696 126.445381 \r\n",
       "L 297.437117 156.530196 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_4\">\r\n",
       "    <g id=\"line2d_6\">\r\n",
       "     <path d=\"M 219.538974 207.033578 \r\n",
       "L 223.62902 207.897033 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_6\">\r\n",
       "     <!-- −1 -->\r\n",
       "     <g transform=\"translate(226.965475 226.130052)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "      <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_5\">\r\n",
       "    <g id=\"line2d_7\">\r\n",
       "     <path d=\"M 259.138973 180.794545 \r\n",
       "L 263.132719 181.60032 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_7\">\r\n",
       "     <!-- 0 -->\r\n",
       "     <g transform=\"translate(270.245112 199.505772)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_6\">\r\n",
       "    <g id=\"line2d_8\">\r\n",
       "     <path d=\"M 296.137666 156.279142 \r\n",
       "L 300.038731 157.032828 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_8\">\r\n",
       "     <!-- 1 -->\r\n",
       "     <g transform=\"translate(306.767603 174.630097)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       "  <g id=\"axis3d_3\">\r\n",
       "   <g id=\"line2d_9\">\r\n",
       "    <path d=\"M 302.626265 153.086553 \r\n",
       "L 309.120561 48.145466 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"Line3DCollection_3\">\r\n",
       "    <path d=\"M 302.749786 151.090589 \r\n",
       "L 147.374017 121.340379 \r\n",
       "L 51.451176 174.556388 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "    <path d=\"M 305.790226 101.960237 \r\n",
       "L 146.626329 73.59687 \r\n",
       "L 48.205767 124.373036 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "    <path d=\"M 308.983989 50.352332 \r\n",
       "L 145.842576 23.550504 \r\n",
       "L 44.791059 71.571851 \r\n",
       "\" style=\"fill:none;stroke:#b0b0b0;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_7\">\r\n",
       "    <g id=\"line2d_10\">\r\n",
       "     <path d=\"M 301.453397 150.842366 \r\n",
       "L 305.34526 151.587551 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_9\">\r\n",
       "     <!-- −1 -->\r\n",
       "     <g transform=\"translate(314.787552 156.876663)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "      <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_8\">\r\n",
       "    <g id=\"line2d_11\">\r\n",
       "     <path d=\"M 304.460886 101.723345 \r\n",
       "L 308.451738 102.434525 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_10\">\r\n",
       "     <!-- 0 -->\r\n",
       "     <g transform=\"translate(322.486062 107.82914)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"xtick_9\">\r\n",
       "    <g id=\"line2d_12\">\r\n",
       "     <path d=\"M 307.619981 50.128245 \r\n",
       "L 311.714985 50.800996 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-width:0.8;\"/>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_11\">\r\n",
       "     <!-- 1 -->\r\n",
       "     <g transform=\"translate(326.171195 56.31481)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       "  <g id=\"axes_1\">\r\n",
       "   <g id=\"line2d_13\">\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 0 5 \r\n",
       "C 1.326016 5 2.597899 4.473168 3.535534 3.535534 \r\n",
       "C 4.473168 2.597899 5 1.326016 5 0 \r\n",
       "C 5 -1.326016 4.473168 -2.597899 3.535534 -3.535534 \r\n",
       "C 2.597899 -4.473168 1.326016 -5 0 -5 \r\n",
       "C -1.326016 -5 -2.597899 -4.473168 -3.535534 -3.535534 \r\n",
       "C -4.473168 -2.597899 -5 -1.326016 -5 0 \r\n",
       "C -5 1.326016 -4.473168 2.597899 -3.535534 3.535534 \r\n",
       "C -2.597899 4.473168 -1.326016 5 0 5 \r\n",
       "z\r\n",
       "\" id=\"m12303e23eb\" style=\"stroke:#ff0000;\"/>\r\n",
       "    </defs>\r\n",
       "    <g clip-path=\"url(#pfa3254ba26)\">\r\n",
       "     <use style=\"fill:#ff0000;stroke:#ff0000;\" x=\"179.124324\" xlink:href=\"#m12303e23eb\" y=\"112.981622\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"Line3DCollection_4\">\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 65.111718 122.902025 \r\n",
       "L 67.786559 114.683274 \r\n",
       "L 70.491104 106.909922 \r\n",
       "L 73.223586 99.588225 \r\n",
       "L 75.982196 92.723846 \r\n",
       "L 78.765084 86.321833 \r\n",
       "L 81.570369 80.386614 \r\n",
       "L 84.396139 74.92198 \r\n",
       "L 87.240454 69.931083 \r\n",
       "L 90.101357 65.416427 \r\n",
       "L 92.97687 61.379862 \r\n",
       "L 95.865006 57.822589 \r\n",
       "L 98.763769 54.745155 \r\n",
       "L 101.671161 52.14746 \r\n",
       "L 104.585182 50.028757 \r\n",
       "L 107.503842 48.387665 \r\n",
       "L 110.425159 47.222175 \r\n",
       "L 113.347166 46.529662 \r\n",
       "L 116.267913 46.306897 \r\n",
       "L 119.185476 46.550066 \r\n",
       "L 122.097955 47.254779 \r\n",
       "L 125.003482 48.416098 \r\n",
       "L 127.900221 50.028547 \r\n",
       "L 130.786376 52.086142 \r\n",
       "L 133.660189 54.582404 \r\n",
       "L 136.519949 57.510393 \r\n",
       "L 139.363988 60.862722 \r\n",
       "L 142.190689 64.631589 \r\n",
       "L 144.998486 68.808801 \r\n",
       "L 147.785867 73.385798 \r\n",
       "L 150.551375 78.353685 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 75.128833 137.471677 \r\n",
       "L 77.80356 129.296847 \r\n",
       "L 80.505629 121.562578 \r\n",
       "L 83.233275 114.275069 \r\n",
       "L 85.984695 107.439929 \r\n",
       "L 88.75805 101.062166 \r\n",
       "L 91.55147 95.146172 \r\n",
       "L 94.36306 89.695715 \r\n",
       "L 97.190901 84.713932 \r\n",
       "L 100.033055 80.20332 \r\n",
       "L 102.887573 76.165735 \r\n",
       "L 105.752495 72.602387 \r\n",
       "L 108.625856 69.513847 \r\n",
       "L 111.505691 66.900043 \r\n",
       "L 114.39004 64.76027 \r\n",
       "L 117.27695 63.093194 \r\n",
       "L 120.164481 61.896863 \r\n",
       "L 123.050711 61.168715 \r\n",
       "L 125.933735 60.905595 \r\n",
       "L 128.811678 61.103766 \r\n",
       "L 131.682689 61.75893 \r\n",
       "L 134.544953 62.866238 \r\n",
       "L 137.396686 64.420319 \r\n",
       "L 140.236147 66.415291 \r\n",
       "L 143.061635 68.844792 \r\n",
       "L 145.871494 71.701995 \r\n",
       "L 148.664115 74.979639 \r\n",
       "L 151.437939 78.670045 \r\n",
       "L 154.19146 82.765153 \r\n",
       "L 156.923226 87.256536 \r\n",
       "L 159.631838 92.135435 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 84.98906 150.146284 \r\n",
       "L 87.665922 142.005946 \r\n",
       "L 90.367842 134.302171 \r\n",
       "L 93.093055 127.04111 \r\n",
       "L 95.83976 120.228333 \r\n",
       "L 98.606125 113.868816 \r\n",
       "L 101.39029 107.966926 \r\n",
       "L 104.19037 102.526414 \r\n",
       "L 107.004464 97.550406 \r\n",
       "L 109.830653 93.041395 \r\n",
       "L 112.667007 89.001241 \r\n",
       "L 115.511593 85.431168 \r\n",
       "L 118.362472 82.331764 \r\n",
       "L 121.21771 79.702984 \r\n",
       "L 124.075378 77.544155 \r\n",
       "L 126.93356 75.853985 \r\n",
       "L 129.790353 74.630565 \r\n",
       "L 132.643873 73.871391 \r\n",
       "L 135.49226 73.573364 \r\n",
       "L 138.333681 73.732815 \r\n",
       "L 141.166333 74.345515 \r\n",
       "L 143.988445 75.406693 \r\n",
       "L 146.798286 76.911059 \r\n",
       "L 149.594164 78.85282 \r\n",
       "L 152.37443 81.225703 \r\n",
       "L 155.137481 84.022977 \r\n",
       "L 157.881763 87.23748 \r\n",
       "L 160.605773 90.861637 \r\n",
       "L 163.308059 94.887492 \r\n",
       "L 165.987225 99.306729 \r\n",
       "L 168.641933 104.110697 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 94.720141 160.975356 \r\n",
       "L 97.401403 152.8603 \r\n",
       "L 100.105502 145.178616 \r\n",
       "L 102.830672 137.936424 \r\n",
       "L 105.575112 131.139266 \r\n",
       "L 108.336993 124.792094 \r\n",
       "L 111.114459 118.899259 \r\n",
       "L 113.905638 113.464501 \r\n",
       "L 116.708636 108.490939 \r\n",
       "L 119.521553 103.981067 \r\n",
       "L 122.342475 99.936751 \r\n",
       "L 125.16949 96.359224 \r\n",
       "L 128.000682 93.249092 \r\n",
       "L 130.834145 90.606332 \r\n",
       "L 133.667978 88.430298 \r\n",
       "L 136.500294 86.719729 \r\n",
       "L 139.329226 85.472757 \r\n",
       "L 142.152927 84.686917 \r\n",
       "L 144.969573 84.359159 \r\n",
       "L 147.777371 84.485865 \r\n",
       "L 150.574561 85.062863 \r\n",
       "L 153.359417 86.085443 \r\n",
       "L 156.130253 87.548377 \r\n",
       "L 158.885424 89.445941 \r\n",
       "L 161.623329 91.771933 \r\n",
       "L 164.342417 94.519698 \r\n",
       "L 167.041183 97.682149 \r\n",
       "L 169.718177 101.251792 \r\n",
       "L 172.372001 105.220753 \r\n",
       "L 175.001311 109.5808 \r\n",
       "L 177.604823 114.32337 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 104.348875 169.998059 \r\n",
       "L 107.036824 161.899229 \r\n",
       "L 109.745445 154.23137 \r\n",
       "L 112.472964 147.00058 \r\n",
       "L 115.217577 140.212385 \r\n",
       "L 117.977454 133.871724 \r\n",
       "L 120.750743 127.982941 \r\n",
       "L 123.535577 122.549769 \r\n",
       "L 126.330072 117.575326 \r\n",
       "L 129.132337 113.06211 \r\n",
       "L 131.940474 109.011992 \r\n",
       "L 134.752587 105.426218 \r\n",
       "L 137.566782 102.305405 \r\n",
       "L 140.381172 99.64955 \r\n",
       "L 143.193884 97.458028 \r\n",
       "L 146.003058 95.729605 \r\n",
       "L 148.806857 94.462439 \r\n",
       "L 151.603465 93.654099 \r\n",
       "L 154.391094 93.301572 \r\n",
       "L 157.167989 93.401276 \r\n",
       "L 159.932427 93.949081 \r\n",
       "L 162.682724 94.940321 \r\n",
       "L 165.417234 96.369817 \r\n",
       "L 168.134359 98.231893 \r\n",
       "L 170.832543 100.5204 \r\n",
       "L 173.51028 103.228735 \r\n",
       "L 176.166117 106.34987 \r\n",
       "L 178.79865 109.876367 \r\n",
       "L 181.406533 113.800412 \r\n",
       "L 183.988475 118.113834 \r\n",
       "L 186.543241 122.808132 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 113.901363 177.243654 \r\n",
       "L 116.598325 169.152092 \r\n",
       "L 119.313833 161.489872 \r\n",
       "L 122.046104 154.263084 \r\n",
       "L 124.793328 147.477249 \r\n",
       "L 127.553672 141.137303 \r\n",
       "L 130.325284 135.247588 \r\n",
       "L 133.106297 129.811838 \r\n",
       "L 135.894834 124.833177 \r\n",
       "L 138.68901 120.314106 \r\n",
       "L 141.48694 116.256505 \r\n",
       "L 144.286741 112.66163 \r\n",
       "L 147.086534 109.530111 \r\n",
       "L 149.884453 106.861959 \r\n",
       "L 152.678644 104.656565 \r\n",
       "L 155.467275 102.912712 \r\n",
       "L 158.248533 101.628582 \r\n",
       "L 161.020632 100.801763 \r\n",
       "L 163.781816 100.429266 \r\n",
       "L 166.530362 100.507537 \r\n",
       "L 169.264584 101.032472 \r\n",
       "L 171.982835 101.999434 \r\n",
       "L 174.68351 103.403274 \r\n",
       "L 177.365051 105.238349 \r\n",
       "L 180.025945 107.498542 \r\n",
       "L 182.664732 110.177285 \r\n",
       "L 185.280004 113.267584 \r\n",
       "L 187.870404 116.762039 \r\n",
       "L 190.434634 120.652872 \r\n",
       "L 192.971452 124.931951 \r\n",
       "L 195.479676 129.590812 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 123.403245 182.73181 \r\n",
       "L 126.111593 174.638597 \r\n",
       "L 128.836391 166.973861 \r\n",
       "L 131.575845 159.743697 \r\n",
       "L 134.328134 152.953631 \r\n",
       "L 137.091418 146.608606 \r\n",
       "L 139.863842 140.71297 \r\n",
       "L 142.643537 135.270467 \r\n",
       "L 145.428628 130.284225 \r\n",
       "L 148.217235 125.756757 \r\n",
       "L 151.007481 121.689952 \r\n",
       "L 153.797492 118.085075 \r\n",
       "L 156.585403 114.942766 \r\n",
       "L 159.369363 112.263047 \r\n",
       "L 162.147538 110.045321 \r\n",
       "L 164.918114 108.288382 \r\n",
       "L 167.679304 106.990423 \r\n",
       "L 170.429347 106.149046 \r\n",
       "L 173.166516 105.761275 \r\n",
       "L 175.889119 105.823567 \r\n",
       "L 178.595503 106.331833 \r\n",
       "L 181.284054 107.28145 \r\n",
       "L 183.953207 108.667282 \r\n",
       "L 186.601441 110.4837 \r\n",
       "L 189.227285 112.724602 \r\n",
       "L 191.829321 115.383434 \r\n",
       "L 194.406183 118.453218 \r\n",
       "L 196.956561 121.926569 \r\n",
       "L 199.479204 125.795724 \r\n",
       "L 201.972917 130.052565 \r\n",
       "L 204.436567 134.688646 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 132.879933 186.472779 \r\n",
       "L 135.602101 178.36898 \r\n",
       "L 138.338642 170.693553 \r\n",
       "L 141.087746 163.452611 \r\n",
       "L 143.84758 156.651698 \r\n",
       "L 146.616293 150.295773 \r\n",
       "L 149.392023 144.389197 \r\n",
       "L 152.172896 138.935729 \r\n",
       "L 154.957036 133.938512 \r\n",
       "L 157.742565 129.400067 \r\n",
       "L 160.527609 125.322296 \r\n",
       "L 163.3103 121.706473 \r\n",
       "L 166.088784 118.553247 \r\n",
       "L 168.861223 115.862646 \r\n",
       "L 171.625798 113.634079 \r\n",
       "L 174.380713 111.866347 \r\n",
       "L 177.1242 110.557645 \r\n",
       "L 179.854524 109.705578 \r\n",
       "L 182.569983 109.30717 \r\n",
       "L 185.268911 109.358882 \r\n",
       "L 187.949686 109.856621 \r\n",
       "L 190.61073 110.795764 \r\n",
       "L 193.250508 112.171173 \r\n",
       "L 195.867539 113.977215 \r\n",
       "L 198.46039 116.207784 \r\n",
       "L 201.027682 118.856322 \r\n",
       "L 203.568092 121.915844 \r\n",
       "L 206.080356 125.378959 \r\n",
       "L 208.563265 129.237898 \r\n",
       "L 211.015671 133.484536 \r\n",
       "L 213.436489 138.110419 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 142.356832 188.467451 \r\n",
       "L 145.095329 180.344054 \r\n",
       "L 147.846131 172.64969 \r\n",
       "L 150.607406 165.390502 \r\n",
       "L 153.377305 158.572062 \r\n",
       "L 156.153965 152.199354 \r\n",
       "L 158.935513 146.276764 \r\n",
       "L 161.720067 140.80807 \r\n",
       "L 164.505746 135.796431 \r\n",
       "L 167.290669 131.244387 \r\n",
       "L 170.072963 127.15385 \r\n",
       "L 172.850766 123.526102 \r\n",
       "L 175.622229 120.3618 \r\n",
       "L 178.385522 117.660976 \r\n",
       "L 181.13884 115.42304 \r\n",
       "L 183.880402 113.64679 \r\n",
       "L 186.608459 112.330416 \r\n",
       "L 189.321294 111.471517 \r\n",
       "L 192.017228 111.067108 \r\n",
       "L 194.694624 111.113636 \r\n",
       "L 197.351886 111.606994 \r\n",
       "L 199.987466 112.542543 \r\n",
       "L 202.599863 113.915124 \r\n",
       "L 205.187629 115.719085 \r\n",
       "L 207.749369 117.948297 \r\n",
       "L 210.283744 120.596177 \r\n",
       "L 212.789472 123.655713 \r\n",
       "L 215.265329 127.119489 \r\n",
       "L 217.710151 130.979705 \r\n",
       "L 220.122837 135.228206 \r\n",
       "L 222.502346 139.856508 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 151.859577 188.707276 \r\n",
       "L 154.617001 180.555137 \r\n",
       "L 157.384656 172.833466 \r\n",
       "L 160.160688 165.548449 \r\n",
       "L 162.94323 158.705698 \r\n",
       "L 165.730399 152.310233 \r\n",
       "L 168.520307 146.366471 \r\n",
       "L 171.311065 140.878216 \r\n",
       "L 174.100779 135.848652 \r\n",
       "L 176.887566 131.280336 \r\n",
       "L 179.66955 127.175192 \r\n",
       "L 182.444867 123.534514 \r\n",
       "L 185.211674 120.358962 \r\n",
       "L 187.968147 117.648567 \r\n",
       "L 190.71249 115.402737 \r\n",
       "L 193.442934 113.620259 \r\n",
       "L 196.157746 112.299313 \r\n",
       "L 198.855226 111.437477 \r\n",
       "L 201.533718 111.031748 \r\n",
       "L 204.191606 111.078545 \r\n",
       "L 206.82732 111.573734 \r\n",
       "L 209.439342 112.512643 \r\n",
       "L 212.026202 113.890079 \r\n",
       "L 214.586485 115.700348 \r\n",
       "L 217.118832 117.937279 \r\n",
       "L 219.62194 120.594248 \r\n",
       "L 222.094568 123.664193 \r\n",
       "L 224.535532 127.139649 \r\n",
       "L 226.943714 131.012765 \r\n",
       "L 229.318056 135.275333 \r\n",
       "L 231.657564 139.918814 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 161.41426 187.174063 \r\n",
       "L 164.193314 178.983844 \r\n",
       "L 166.98051 171.226318 \r\n",
       "L 169.773969 163.907728 \r\n",
       "L 172.571797 157.033738 \r\n",
       "L 175.372094 150.609413 \r\n",
       "L 178.172954 144.639211 \r\n",
       "L 180.97247 139.12697 \r\n",
       "L 183.76874 134.075901 \r\n",
       "L 186.559869 129.488582 \r\n",
       "L 189.343976 125.366954 \r\n",
       "L 192.119196 121.712318 \r\n",
       "L 194.883685 118.525337 \r\n",
       "L 197.635623 115.806041 \r\n",
       "L 200.37322 113.553824 \r\n",
       "L 203.094718 111.76746 \r\n",
       "L 205.798394 110.445105 \r\n",
       "L 208.482568 109.584313 \r\n",
       "L 211.145598 109.182044 \r\n",
       "L 213.785892 109.23468 \r\n",
       "L 216.401904 109.738045 \r\n",
       "L 218.992142 110.687416 \r\n",
       "L 221.555167 112.077547 \r\n",
       "L 224.089595 113.902688 \r\n",
       "L 226.594101 116.156607 \r\n",
       "L 229.06742 118.832611 \r\n",
       "L 231.508348 121.923573 \r\n",
       "L 233.915745 125.421954 \r\n",
       "L 236.288533 129.31983 \r\n",
       "L 238.6257 133.608915 \r\n",
       "L 240.926297 138.280592 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 171.047683 183.839641 \r\n",
       "L 173.851192 175.601745 \r\n",
       "L 176.660726 167.79958 \r\n",
       "L 179.474377 160.439461 \r\n",
       "L 182.290224 153.527116 \r\n",
       "L 185.106341 147.067665 \r\n",
       "L 187.920802 141.065617 \r\n",
       "L 190.731681 135.524849 \r\n",
       "L 193.537059 130.448606 \r\n",
       "L 196.335032 125.839492 \r\n",
       "L 199.123707 121.699462 \r\n",
       "L 201.901214 118.029828 \r\n",
       "L 204.665706 114.831253 \r\n",
       "L 207.415364 112.103759 \r\n",
       "L 210.148402 109.846725 \r\n",
       "L 212.863069 108.058901 \r\n",
       "L 215.557653 106.738413 \r\n",
       "L 218.230486 105.882776 \r\n",
       "L 220.879944 105.488905 \r\n",
       "L 223.504453 105.553129 \r\n",
       "L 226.102492 106.071213 \r\n",
       "L 228.672593 107.038369 \r\n",
       "L 231.213344 108.44928 \r\n",
       "L 233.723394 110.298118 \r\n",
       "L 236.201451 112.57857 \r\n",
       "L 238.646287 115.283856 \r\n",
       "L 241.056735 118.406757 \r\n",
       "L 243.431697 121.93964 \r\n",
       "L 245.770136 125.874482 \r\n",
       "L 248.071086 130.202895 \r\n",
       "L 250.333645 134.916156 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 180.787616 178.66537 \r\n",
       "L 183.618544 170.369875 \r\n",
       "L 186.453342 162.513993 \r\n",
       "L 189.290066 155.104124 \r\n",
       "L 192.126766 148.146072 \r\n",
       "L 194.961486 141.645029 \r\n",
       "L 197.792274 135.605557 \r\n",
       "L 200.617181 130.031586 \r\n",
       "L 203.434272 124.926396 \r\n",
       "L 206.241623 120.292621 \r\n",
       "L 209.037332 116.132235 \r\n",
       "L 211.819518 112.446557 \r\n",
       "L 214.586328 109.236251 \r\n",
       "L 217.335943 106.501323 \r\n",
       "L 220.066576 104.241133 \r\n",
       "L 222.776481 102.454401 \r\n",
       "L 225.463954 101.139211 \r\n",
       "L 228.127339 100.293027 \r\n",
       "L 230.765026 99.912707 \r\n",
       "L 233.37546 99.994514 \r\n",
       "L 235.95714 100.534135 \r\n",
       "L 238.508623 101.5267 \r\n",
       "L 241.028525 102.966802 \r\n",
       "L 243.515524 104.848516 \r\n",
       "L 245.968363 107.165424 \r\n",
       "L 248.385847 109.910639 \r\n",
       "L 250.76685 113.076825 \r\n",
       "L 253.110312 116.656231 \r\n",
       "L 255.415243 120.640707 \r\n",
       "L 257.680719 125.02174 \r\n",
       "L 259.905888 129.790475 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 190.663078 171.601505 \r\n",
       "L 193.524552 163.238088 \r\n",
       "L 196.387688 155.319047 \r\n",
       "L 199.250503 147.850883 \r\n",
       "L 202.111011 140.839492 \r\n",
       "L 204.967223 134.290144 \r\n",
       "L 207.817157 128.20747 \r\n",
       "L 210.658839 122.595455 \r\n",
       "L 213.490308 117.457426 \r\n",
       "L 216.309624 112.796046 \r\n",
       "L 219.114865 108.613312 \r\n",
       "L 221.904141 104.910549 \r\n",
       "L 224.67559 101.688417 \r\n",
       "L 227.427386 98.946907 \r\n",
       "L 230.157741 96.685351 \r\n",
       "L 232.864913 94.902428 \r\n",
       "L 235.547202 93.596171 \r\n",
       "L 238.202961 92.763984 \r\n",
       "L 240.830595 92.402651 \r\n",
       "L 243.428565 92.508351 \r\n",
       "L 245.995389 93.076681 \r\n",
       "L 248.529648 94.102669 \r\n",
       "L 251.029985 95.580795 \r\n",
       "L 253.495108 97.505017 \r\n",
       "L 255.923792 99.86879 \r\n",
       "L 258.314879 102.665092 \r\n",
       "L 260.667281 105.88645 \r\n",
       "L 262.97998 109.524965 \r\n",
       "L 265.25203 113.572337 \r\n",
       "L 267.482552 118.019897 \r\n",
       "L 269.670744 122.858631 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 200.704653 162.586365 \r\n",
       "L 203.599985 154.144224 \r\n",
       "L 206.494704 146.152148 \r\n",
       "L 209.386784 138.616759 \r\n",
       "L 212.274196 131.544056 \r\n",
       "L 215.154916 124.939403 \r\n",
       "L 218.026926 118.807509 \r\n",
       "L 220.888222 113.152423 \r\n",
       "L 223.736817 107.977523 \r\n",
       "L 226.570745 103.285509 \r\n",
       "L 229.388067 99.078398 \r\n",
       "L 232.186876 95.357524 \r\n",
       "L 234.965297 92.123539 \r\n",
       "L 237.721497 89.376413 \r\n",
       "L 240.453685 87.115443 \r\n",
       "L 243.160117 85.33926 \r\n",
       "L 245.839098 84.045835 \r\n",
       "L 248.488988 83.232496 \r\n",
       "L 251.108204 82.895939 \r\n",
       "L 253.695222 83.032245 \r\n",
       "L 256.248581 83.636899 \r\n",
       "L 258.766882 84.704805 \r\n",
       "L 261.248795 86.230313 \r\n",
       "L 263.693059 88.207238 \r\n",
       "L 266.098481 90.628884 \r\n",
       "L 268.46394 93.48807 \r\n",
       "L 270.788388 96.777156 \r\n",
       "L 273.070848 100.488069 \r\n",
       "L 275.310419 104.612329 \r\n",
       "L 277.506272 109.141083 \r\n",
       "L 279.657652 114.065127 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 210.94483 151.545314 \r\n",
       "L 213.877544 143.013076 \r\n",
       "L 216.807288 134.937577 \r\n",
       "L 219.731987 127.325574 \r\n",
       "L 222.649565 120.183191 \r\n",
       "L 225.557956 113.515893 \r\n",
       "L 228.455103 107.328483 \r\n",
       "L 231.338967 101.625082 \r\n",
       "L 234.20753 96.409124 \r\n",
       "L 237.058797 91.68335 \r\n",
       "L 239.890808 87.4498 \r\n",
       "L 242.701634 83.709816 \r\n",
       "L 245.489387 80.464038 \r\n",
       "L 248.252222 77.712412 \r\n",
       "L 250.988341 75.45419 \r\n",
       "L 253.695998 73.687945 \r\n",
       "L 256.3735 72.411574 \r\n",
       "L 259.019214 71.622315 \r\n",
       "L 261.631566 71.31676 \r\n",
       "L 264.209046 71.490873 \r\n",
       "L 266.750211 72.140006 \r\n",
       "L 269.253687 73.25892 \r\n",
       "L 271.718169 74.841809 \r\n",
       "L 274.142425 76.882319 \r\n",
       "L 276.525297 79.373579 \r\n",
       "L 278.865701 82.308218 \r\n",
       "L 281.162629 85.678402 \r\n",
       "L 283.415149 89.475853 \r\n",
       "L 285.622407 93.691882 \r\n",
       "L 287.783622 98.317418 \r\n",
       "L 289.898093 103.343034 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 65.111718 122.902025 \r\n",
       "L 70.141665 130.427245 \r\n",
       "L 75.128833 137.471677 \r\n",
       "L 80.076799 144.042518 \r\n",
       "L 84.98906 150.146284 \r\n",
       "L 89.86905 155.788827 \r\n",
       "L 94.720141 160.975356 \r\n",
       "L 99.545657 165.710447 \r\n",
       "L 104.348875 169.998059 \r\n",
       "L 109.133039 173.841543 \r\n",
       "L 113.901363 177.243654 \r\n",
       "L 118.657039 180.206551 \r\n",
       "L 123.403245 182.73181 \r\n",
       "L 128.143153 184.820417 \r\n",
       "L 132.879933 186.472779 \r\n",
       "L 137.616762 187.688714 \r\n",
       "L 142.356832 188.467451 \r\n",
       "L 147.103356 188.807627 \r\n",
       "L 151.859577 188.707276 \r\n",
       "L 156.62877 188.163821 \r\n",
       "L 161.41426 187.174063 \r\n",
       "L 166.21942 185.734166 \r\n",
       "L 171.047683 183.839641 \r\n",
       "L 175.902555 181.485326 \r\n",
       "L 180.787616 178.66537 \r\n",
       "L 185.706535 175.373201 \r\n",
       "L 190.663078 171.601505 \r\n",
       "L 195.66112 167.342192 \r\n",
       "L 200.704653 162.586365 \r\n",
       "L 205.7978 157.324282 \r\n",
       "L 210.94483 151.545314 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 70.491104 106.909922 \r\n",
       "L 75.519183 114.478927 \r\n",
       "L 80.505629 121.562578 \r\n",
       "L 85.454015 128.168136 \r\n",
       "L 90.367842 134.302171 \r\n",
       "L 95.250546 139.970582 \r\n",
       "L 100.105502 145.178616 \r\n",
       "L 104.93604 149.930884 \r\n",
       "L 109.745445 154.23137 \r\n",
       "L 114.536968 158.083444 \r\n",
       "L 119.313833 161.489872 \r\n",
       "L 124.079243 164.45282 \r\n",
       "L 128.836391 166.973861 \r\n",
       "L 133.588462 169.053975 \r\n",
       "L 138.338642 170.693553 \r\n",
       "L 143.090128 171.892391 \r\n",
       "L 147.846131 172.64969 \r\n",
       "L 152.609885 172.964051 \r\n",
       "L 157.384656 172.833466 \r\n",
       "L 162.173748 172.255307 \r\n",
       "L 166.98051 171.226318 \r\n",
       "L 171.808347 169.742596 \r\n",
       "L 176.660726 167.79958 \r\n",
       "L 181.541184 165.392027 \r\n",
       "L 186.453342 162.513993 \r\n",
       "L 191.400906 159.158807 \r\n",
       "L 196.387688 155.319047 \r\n",
       "L 201.417606 150.986503 \r\n",
       "L 206.494704 146.152148 \r\n",
       "L 211.623157 140.8061 \r\n",
       "L 216.807288 134.937577 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 75.982196 92.723846 \r\n",
       "L 81.003649 100.326413 \r\n",
       "L 85.984695 107.439929 \r\n",
       "L 90.9289 114.071699 \r\n",
       "L 95.83976 120.228333 \r\n",
       "L 100.720706 125.915763 \r\n",
       "L 105.575112 131.139266 \r\n",
       "L 110.406307 135.903472 \r\n",
       "L 115.217577 140.212385 \r\n",
       "L 120.012174 144.069386 \r\n",
       "L 124.793328 147.477249 \r\n",
       "L 129.564249 150.438141 \r\n",
       "L 134.328134 152.953631 \r\n",
       "L 139.088178 155.024691 \r\n",
       "L 143.84758 156.651698 \r\n",
       "L 148.609547 157.834429 \r\n",
       "L 153.377305 158.572062 \r\n",
       "L 158.154105 158.863166 \r\n",
       "L 162.94323 158.705698 \r\n",
       "L 167.748003 158.09699 \r\n",
       "L 172.571797 157.033738 \r\n",
       "L 177.41804 155.511987 \r\n",
       "L 182.290224 153.527116 \r\n",
       "L 187.191916 151.073816 \r\n",
       "L 192.126766 148.146072 \r\n",
       "L 197.098516 144.737135 \r\n",
       "L 202.111011 140.839492 \r\n",
       "L 207.16821 136.444843 \r\n",
       "L 212.274196 131.544056 \r\n",
       "L 217.433191 126.127139 \r\n",
       "L 222.649565 120.183191 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 81.570369 80.386614 \r\n",
       "L 86.580473 88.012322 \r\n",
       "L 91.55147 95.146172 \r\n",
       "L 96.486915 101.795495 \r\n",
       "L 101.39029 107.966926 \r\n",
       "L 106.265016 113.666417 \r\n",
       "L 111.114459 118.899259 \r\n",
       "L 115.941941 123.670097 \r\n",
       "L 120.750743 127.982941 \r\n",
       "L 125.544116 131.84118 \r\n",
       "L 130.325284 135.247588 \r\n",
       "L 135.09746 138.20433 \r\n",
       "L 139.863842 140.71297 \r\n",
       "L 144.627628 142.774472 \r\n",
       "L 149.392023 144.389197 \r\n",
       "L 154.160239 145.556909 \r\n",
       "L 158.935513 146.276764 \r\n",
       "L 163.721104 146.547308 \r\n",
       "L 168.520307 146.366471 \r\n",
       "L 173.336462 145.731551 \r\n",
       "L 178.172954 144.639211 \r\n",
       "L 183.03323 143.085454 \r\n",
       "L 187.920802 141.065617 \r\n",
       "L 192.839259 138.574341 \r\n",
       "L 197.792274 135.605557 \r\n",
       "L 202.783615 132.152458 \r\n",
       "L 207.817157 128.20747 \r\n",
       "L 212.896888 123.762221 \r\n",
       "L 218.026926 118.807509 \r\n",
       "L 223.211528 113.333258 \r\n",
       "L 228.455103 107.328483 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 87.240454 69.931083 \r\n",
       "L 92.234546 77.569388 \r\n",
       "L 97.190901 84.713932 \r\n",
       "L 102.113052 91.372058 \r\n",
       "L 107.004464 97.550406 \r\n",
       "L 111.868541 103.254934 \r\n",
       "L 116.708636 108.490939 \r\n",
       "L 121.528056 113.263066 \r\n",
       "L 126.330072 117.575326 \r\n",
       "L 131.117925 121.431107 \r\n",
       "L 135.894834 124.833177 \r\n",
       "L 140.664002 127.783698 \r\n",
       "L 145.428628 130.284225 \r\n",
       "L 150.191905 132.335713 \r\n",
       "L 154.957036 133.938512 \r\n",
       "L 159.727237 135.092371 \r\n",
       "L 164.505746 135.796431 \r\n",
       "L 169.295826 136.049222 \r\n",
       "L 174.100779 135.848652 \r\n",
       "L 178.923952 135.192 \r\n",
       "L 183.76874 134.075901 \r\n",
       "L 188.6386 132.496335 \r\n",
       "L 193.537059 130.448606 \r\n",
       "L 198.46772 127.927328 \r\n",
       "L 203.434272 124.926396 \r\n",
       "L 208.440503 121.438967 \r\n",
       "L 213.490308 117.457426 \r\n",
       "L 218.587699 112.973361 \r\n",
       "L 223.736817 107.977523 \r\n",
       "L 228.941947 102.459791 \r\n",
       "L 234.20753 96.409124 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 92.97687 61.379862 \r\n",
       "L 97.950375 69.020173 \r\n",
       "L 102.887573 76.165735 \r\n",
       "L 107.791971 82.82388 \r\n",
       "L 112.667007 89.001241 \r\n",
       "L 117.516065 94.703769 \r\n",
       "L 122.342475 99.936751 \r\n",
       "L 127.149527 104.704824 \r\n",
       "L 131.940474 109.011992 \r\n",
       "L 136.718543 112.861632 \r\n",
       "L 141.48694 116.256505 \r\n",
       "L 146.248857 119.198763 \r\n",
       "L 151.007481 121.689952 \r\n",
       "L 155.766 123.731016 \r\n",
       "L 160.527609 125.322296 \r\n",
       "L 165.295518 126.463531 \r\n",
       "L 170.072963 127.15385 \r\n",
       "L 174.863207 127.391771 \r\n",
       "L 179.66955 127.175192 \r\n",
       "L 184.49534 126.501378 \r\n",
       "L 189.343976 125.366954 \r\n",
       "L 194.218921 123.767884 \r\n",
       "L 199.123707 121.699462 \r\n",
       "L 204.061944 119.156285 \r\n",
       "L 209.037332 116.132235 \r\n",
       "L 214.05367 112.620453 \r\n",
       "L 219.114865 108.613312 \r\n",
       "L 224.224945 104.102381 \r\n",
       "L 229.388067 99.078398 \r\n",
       "L 234.608535 93.531223 \r\n",
       "L 239.890808 87.4498 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 98.763769 54.745155 \r\n",
       "L 103.712225 62.376914 \r\n",
       "L 108.625856 69.513847 \r\n",
       "L 113.508135 76.163261 \r\n",
       "L 118.362472 82.331764 \r\n",
       "L 123.192219 88.025286 \r\n",
       "L 128.000682 93.249092 \r\n",
       "L 132.791126 98.007804 \r\n",
       "L 137.566782 102.305405 \r\n",
       "L 142.330855 106.145258 \r\n",
       "L 147.086534 109.530111 \r\n",
       "L 151.836993 112.462102 \r\n",
       "L 156.585403 114.942766 \r\n",
       "L 161.334939 116.973037 \r\n",
       "L 166.088784 118.553247 \r\n",
       "L 170.850139 119.683126 \r\n",
       "L 175.622229 120.3618 \r\n",
       "L 180.408308 120.587781 \r\n",
       "L 185.211674 120.358962 \r\n",
       "L 190.035667 119.672606 \r\n",
       "L 194.883685 118.525337 \r\n",
       "L 199.759187 116.913122 \r\n",
       "L 204.665706 114.831253 \r\n",
       "L 209.606852 112.274334 \r\n",
       "L 214.586328 109.236251 \r\n",
       "L 219.607937 105.710151 \r\n",
       "L 224.67559 101.688417 \r\n",
       "L 229.793321 97.16263 \r\n",
       "L 234.965297 92.123539 \r\n",
       "L 240.195829 86.56102 \r\n",
       "L 245.489387 80.464038 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 104.585182 50.028757 \r\n",
       "L 109.504261 57.641513 \r\n",
       "L 114.39004 64.76027 \r\n",
       "L 119.245955 71.392295 \r\n",
       "L 124.075378 77.544155 \r\n",
       "L 128.881628 83.221745 \r\n",
       "L 133.667978 88.430298 \r\n",
       "L 138.437662 93.174406 \r\n",
       "L 143.193884 97.458028 \r\n",
       "L 147.939824 101.284505 \r\n",
       "L 152.678644 104.656565 \r\n",
       "L 157.413499 107.576329 \r\n",
       "L 162.147538 110.045321 \r\n",
       "L 166.883915 112.064462 \r\n",
       "L 171.625798 113.634079 \r\n",
       "L 176.37637 114.753898 \r\n",
       "L 181.13884 115.42304 \r\n",
       "L 185.916453 115.640021 \r\n",
       "L 190.71249 115.402737 \r\n",
       "L 195.530283 114.70846 \r\n",
       "L 200.37322 113.553824 \r\n",
       "L 205.244752 111.934808 \r\n",
       "L 210.148402 109.846725 \r\n",
       "L 215.087778 107.284196 \r\n",
       "L 220.066576 104.241133 \r\n",
       "L 225.088594 100.710714 \r\n",
       "L 230.157741 96.685351 \r\n",
       "L 235.27805 92.156664 \r\n",
       "L 240.453685 87.115443 \r\n",
       "L 245.688958 81.551614 \r\n",
       "L 250.988341 75.45419 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 110.425159 47.222175 \r\n",
       "L 115.310689 54.805658 \r\n",
       "L 120.164481 61.896863 \r\n",
       "L 124.989926 68.502994 \r\n",
       "L 129.790353 74.630565 \r\n",
       "L 134.569041 80.285423 \r\n",
       "L 139.329226 85.472757 \r\n",
       "L 144.074108 90.19712 \r\n",
       "L 148.806857 94.462439 \r\n",
       "L 153.530621 98.272025 \r\n",
       "L 158.248533 101.628582 \r\n",
       "L 162.963719 104.534212 \r\n",
       "L 167.679304 106.990423 \r\n",
       "L 172.398416 108.998128 \r\n",
       "L 177.1242 110.557645 \r\n",
       "L 181.859818 111.668699 \r\n",
       "L 186.608459 112.330416 \r\n",
       "L 191.373346 112.541317 \r\n",
       "L 196.157746 112.299313 \r\n",
       "L 200.964971 111.60169 \r\n",
       "L 205.798394 110.445105 \r\n",
       "L 210.661452 108.825565 \r\n",
       "L 215.557653 106.738413 \r\n",
       "L 220.490593 104.17831 \r\n",
       "L 225.463954 101.139211 \r\n",
       "L 230.481524 97.614341 \r\n",
       "L 235.547202 93.596171 \r\n",
       "L 240.665008 89.076382 \r\n",
       "L 245.839098 84.045835 \r\n",
       "L 251.073775 78.494532 \r\n",
       "L 256.3735 72.411574 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 116.267913 46.306897 \r\n",
       "L 121.115899 53.851091 \r\n",
       "L 125.933735 60.905595 \r\n",
       "L 130.724761 67.477537 \r\n",
       "L 135.49226 73.573364 \r\n",
       "L 140.239466 79.198859 \r\n",
       "L 144.969573 84.359159 \r\n",
       "L 149.685738 89.058768 \r\n",
       "L 154.391094 93.301572 \r\n",
       "L 159.088754 97.090846 \r\n",
       "L 163.781816 100.429266 \r\n",
       "L 168.473372 103.318912 \r\n",
       "L 173.166516 105.761275 \r\n",
       "L 177.864348 107.757256 \r\n",
       "L 182.569983 109.30717 \r\n",
       "L 187.286555 110.410743 \r\n",
       "L 192.017228 111.067108 \r\n",
       "L 196.765202 111.2748 \r\n",
       "L 201.533718 111.031748 \r\n",
       "L 206.326067 110.335266 \r\n",
       "L 211.145598 109.182044 \r\n",
       "L 215.995727 107.568126 \r\n",
       "L 220.879944 105.488905 \r\n",
       "L 225.801822 102.939093 \r\n",
       "L 230.765026 99.912707 \r\n",
       "L 235.773324 96.403044 \r\n",
       "L 240.830595 92.402651 \r\n",
       "L 245.940843 87.903296 \r\n",
       "L 251.108204 82.895939 \r\n",
       "L 256.336964 77.370687 \r\n",
       "L 261.631566 71.31676 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 122.097955 47.254779 \r\n",
       "L 126.904594 54.749985 \r\n",
       "L 131.682689 61.75893 \r\n",
       "L 136.435525 68.288652 \r\n",
       "L 141.166333 74.345515 \r\n",
       "L 145.878296 79.935228 \r\n",
       "L 150.574561 85.062863 \r\n",
       "L 155.258242 89.732869 \r\n",
       "L 159.932427 93.949081 \r\n",
       "L 164.600188 97.714734 \r\n",
       "L 169.264584 101.032472 \r\n",
       "L 173.92867 103.904347 \r\n",
       "L 178.595503 106.331833 \r\n",
       "L 183.268148 108.315821 \r\n",
       "L 187.949686 109.856621 \r\n",
       "L 192.643222 110.953964 \r\n",
       "L 197.351886 111.606994 \r\n",
       "L 202.078848 111.814265 \r\n",
       "L 206.82732 111.573734 \r\n",
       "L 211.600565 110.882751 \r\n",
       "L 216.401904 109.738045 \r\n",
       "L 221.234726 108.135716 \r\n",
       "L 226.102492 106.071213 \r\n",
       "L 231.008751 103.539321 \r\n",
       "L 235.95714 100.534135 \r\n",
       "L 240.951402 97.049039 \r\n",
       "L 245.995389 93.076681 \r\n",
       "L 251.093078 88.608941 \r\n",
       "L 256.248581 83.636899 \r\n",
       "L 261.466153 78.150798 \r\n",
       "L 266.750211 72.140006 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 127.900221 50.028547 \r\n",
       "L 132.661918 57.465445 \r\n",
       "L 137.396686 64.420319 \r\n",
       "L 142.107751 70.900101 \r\n",
       "L 146.798286 76.911059 \r\n",
       "L 151.471422 82.458819 \r\n",
       "L 156.130253 87.548377 \r\n",
       "L 160.777843 92.184116 \r\n",
       "L 165.417234 96.369817 \r\n",
       "L 170.051451 100.108668 \r\n",
       "L 174.68351 103.403274 \r\n",
       "L 179.316423 106.255662 \r\n",
       "L 183.953207 108.667282 \r\n",
       "L 188.596888 110.639016 \r\n",
       "L 193.250508 112.171173 \r\n",
       "L 197.917135 113.263489 \r\n",
       "L 202.599863 113.915124 \r\n",
       "L 207.301826 114.124657 \r\n",
       "L 212.026202 113.890079 \r\n",
       "L 216.77622 113.208781 \r\n",
       "L 221.555167 112.077547 \r\n",
       "L 226.366398 110.492539 \r\n",
       "L 231.213344 108.44928 \r\n",
       "L 236.099518 105.942637 \r\n",
       "L 241.028525 102.966802 \r\n",
       "L 246.004074 99.515266 \r\n",
       "L 251.029985 95.580795 \r\n",
       "L 256.1102 91.155402 \r\n",
       "L 261.248795 86.230313 \r\n",
       "L 266.449992 80.795934 \r\n",
       "L 271.718169 74.841809 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 133.660189 54.582404 \r\n",
       "L 138.373568 61.952107 \r\n",
       "L 143.061635 68.844792 \r\n",
       "L 147.727551 75.26727 \r\n",
       "L 152.37443 81.225703 \r\n",
       "L 157.005343 86.72562 \r\n",
       "L 161.623329 91.771933 \r\n",
       "L 166.231399 96.368954 \r\n",
       "L 170.832543 100.5204 \r\n",
       "L 175.429735 104.229408 \r\n",
       "L 180.025945 107.498542 \r\n",
       "L 184.624138 110.329796 \r\n",
       "L 189.227285 112.724602 \r\n",
       "L 193.838369 114.683828 \r\n",
       "L 198.46039 116.207784 \r\n",
       "L 203.096371 117.296213 \r\n",
       "L 207.749369 117.948297 \r\n",
       "L 212.422477 118.162642 \r\n",
       "L 217.118832 117.937279 \r\n",
       "L 221.841623 117.269653 \r\n",
       "L 226.594101 116.156607 \r\n",
       "L 231.37958 114.594376 \r\n",
       "L 236.201451 112.57857 \r\n",
       "L 241.06319 110.104152 \r\n",
       "L 245.968363 107.165424 \r\n",
       "L 250.920637 103.756002 \r\n",
       "L 255.923792 99.86879 \r\n",
       "L 260.981729 95.495952 \r\n",
       "L 266.098481 90.628884 \r\n",
       "L 271.278226 85.258177 \r\n",
       "L 276.525297 79.373579 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 139.363988 60.862722 \r\n",
       "L 144.025903 68.156825 \r\n",
       "L 148.664115 74.979639 \r\n",
       "L 153.281719 81.337842 \r\n",
       "L 157.881763 87.23748 \r\n",
       "L 162.467258 92.683975 \r\n",
       "L 167.041183 97.682149 \r\n",
       "L 171.606491 102.23623 \r\n",
       "L 176.166117 106.34987 \r\n",
       "L 180.722982 110.026148 \r\n",
       "L 185.280004 113.267584 \r\n",
       "L 189.840096 116.076137 \r\n",
       "L 194.406183 118.453218 \r\n",
       "L 198.981197 120.399684 \r\n",
       "L 203.568092 121.915844 \r\n",
       "L 208.169847 123.001453 \r\n",
       "L 212.789472 123.655713 \r\n",
       "L 217.430014 123.877268 \r\n",
       "L 222.094568 123.664193 \r\n",
       "L 226.786277 123.013989 \r\n",
       "L 231.508348 121.923573 \r\n",
       "L 236.264052 120.389261 \r\n",
       "L 241.056735 118.406757 \r\n",
       "L 245.889828 115.971137 \r\n",
       "L 250.76685 113.076825 \r\n",
       "L 255.691424 109.717577 \r\n",
       "L 260.667281 105.88645 \r\n",
       "L 265.698275 101.575782 \r\n",
       "L 270.788388 96.777156 \r\n",
       "L 275.941746 91.481371 \r\n",
       "L 281.162629 85.678402 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 144.998486 68.808801 \r\n",
       "L 149.606027 76.019419 \r\n",
       "L 154.19146 82.765153 \r\n",
       "L 158.757811 89.052539 \r\n",
       "L 163.308059 94.887492 \r\n",
       "L 167.84515 100.275324 \r\n",
       "L 172.372001 105.220753 \r\n",
       "L 176.891503 109.727924 \r\n",
       "L 181.406533 113.800412 \r\n",
       "L 185.919957 117.441239 \r\n",
       "L 190.434634 120.652872 \r\n",
       "L 194.953427 123.437239 \r\n",
       "L 199.479204 125.795724 \r\n",
       "L 204.014849 127.729174 \r\n",
       "L 208.563265 129.237898 \r\n",
       "L 213.127378 130.321666 \r\n",
       "L 217.710151 130.979705 \r\n",
       "L 222.314582 131.210695 \r\n",
       "L 226.943714 131.012765 \r\n",
       "L 231.600646 130.383479 \r\n",
       "L 236.288533 129.31983 \r\n",
       "L 241.010597 127.818227 \r\n",
       "L 245.770136 125.874482 \r\n",
       "L 250.570528 123.483788 \r\n",
       "L 255.415243 120.640707 \r\n",
       "L 260.307851 117.339147 \r\n",
       "L 265.25203 113.572337 \r\n",
       "L 270.251577 109.332801 \r\n",
       "L 275.310419 104.612329 \r\n",
       "L 280.432622 99.401948 \r\n",
       "L 285.622407 93.691882 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "    <path clip-path=\"url(#pfa3254ba26)\" d=\"M 150.551375 78.353685 \r\n",
       "L 155.101873 85.473487 \r\n",
       "L 159.631838 92.135435 \r\n",
       "L 164.144221 98.345912 \r\n",
       "L 168.641933 104.110697 \r\n",
       "L 173.12785 109.434978 \r\n",
       "L 177.604823 114.32337 \r\n",
       "L 182.075682 118.779921 \r\n",
       "L 186.543241 122.808132 \r\n",
       "L 191.010305 126.410956 \r\n",
       "L 195.479676 129.590812 \r\n",
       "L 199.954159 132.349589 \r\n",
       "L 204.436567 134.688646 \r\n",
       "L 208.929727 136.60882 \r\n",
       "L 213.436489 138.110419 \r\n",
       "L 217.959726 139.19323 \r\n",
       "L 222.502346 139.856508 \r\n",
       "L 227.067295 140.098975 \r\n",
       "L 231.657564 139.918814 \r\n",
       "L 236.276198 139.313661 \r\n",
       "L 240.926297 138.280592 \r\n",
       "L 245.611032 136.816115 \r\n",
       "L 250.333645 134.916156 \r\n",
       "L 255.097459 132.576039 \r\n",
       "L 259.905888 129.790475 \r\n",
       "L 264.762443 126.553535 \r\n",
       "L 269.670744 122.858631 \r\n",
       "L 274.634526 118.69849 \r\n",
       "L 279.657652 114.065127 \r\n",
       "L 284.744124 108.949811 \r\n",
       "L 289.898093 103.343034 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       " </g>\r\n",
       " <defs>\r\n",
       "  <clipPath id=\"pfa3254ba26\">\r\n",
       "   <rect height=\"217.44\" width=\"334.8\" x=\"7.2\" y=\"7.2\"/>\r\n",
       "  </clipPath>\r\n",
       " </defs>\r\n",
       "</svg>\r\n"
      ],
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x, y = np.mgrid[-1: 1: 31j, -1: 1: 31j]\n",
    "z = x**2 - y**2\n",
    "\n",
    "d2l.set_figsize((6, 4))\n",
    "ax = d2l.plt.figure().add_subplot(111, projection='3d')\n",
    "ax.plot_wireframe(x, y, z, **{'rstride': 2, 'cstride': 2})\n",
    "ax.plot([0], [0], [0], 'ro', markersize=10)\n",
    "ticks = [-1,  0, 1]\n",
    "d2l.plt.xticks(ticks)\n",
    "d2l.plt.yticks(ticks)\n",
    "ax.set_zticks(ticks)\n",
    "d2l.plt.xlabel('x')\n",
    "d2l.plt.ylabel('y');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "graffitiCellId": "id_npu7pxn",
    "id": "FFDE2A35B48F448BA7379136660FC940",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 梯度消失"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "graffitiCellId": "id_ypkxce1",
    "id": "FD4109A263F4455EAD8C67C5DC37C027",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(2, 0.0, 'vanishing gradient')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\r\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\r\n",
       "<svg height=\"262.19625pt\" version=\"1.1\" viewBox=\"0 0 400.523437 262.19625\" width=\"400.523437pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n",
       " <defs>\r\n",
       "  <style type=\"text/css\">\r\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\r\n",
       "  </style>\r\n",
       " </defs>\r\n",
       " <g id=\"figure_1\">\r\n",
       "  <g id=\"patch_1\">\r\n",
       "   <path d=\"M 0 262.19625 \r\n",
       "L 400.523437 262.19625 \r\n",
       "L 400.523437 0 \r\n",
       "L 0 0 \r\n",
       "z\r\n",
       "\" style=\"fill:none;\"/>\r\n",
       "  </g>\r\n",
       "  <g id=\"axes_1\">\r\n",
       "   <g id=\"patch_2\">\r\n",
       "    <path d=\"M 58.523438 224.64 \r\n",
       "L 393.323438 224.64 \r\n",
       "L 393.323438 7.2 \r\n",
       "L 58.523438 7.2 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_1\">\r\n",
       "    <g id=\"xtick_1\">\r\n",
       "     <g id=\"line2d_1\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L 0 3.5 \r\n",
       "\" id=\"m54e93e67b9\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"73.741619\" xlink:href=\"#m54e93e67b9\" y=\"224.64\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_1\">\r\n",
       "      <!-- −2 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.59375 35.5 \r\n",
       "L 73.1875 35.5 \r\n",
       "L 73.1875 27.203125 \r\n",
       "L 10.59375 27.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-8722\"/>\r\n",
       "       <path d=\"M 19.1875 8.296875 \r\n",
       "L 53.609375 8.296875 \r\n",
       "L 53.609375 0 \r\n",
       "L 7.328125 0 \r\n",
       "L 7.328125 8.296875 \r\n",
       "Q 12.9375 14.109375 22.625 23.890625 \r\n",
       "Q 32.328125 33.6875 34.8125 36.53125 \r\n",
       "Q 39.546875 41.84375 41.421875 45.53125 \r\n",
       "Q 43.3125 49.21875 43.3125 52.78125 \r\n",
       "Q 43.3125 58.59375 39.234375 62.25 \r\n",
       "Q 35.15625 65.921875 28.609375 65.921875 \r\n",
       "Q 23.96875 65.921875 18.8125 64.3125 \r\n",
       "Q 13.671875 62.703125 7.8125 59.421875 \r\n",
       "L 7.8125 69.390625 \r\n",
       "Q 13.765625 71.78125 18.9375 73 \r\n",
       "Q 24.125 74.21875 28.421875 74.21875 \r\n",
       "Q 39.75 74.21875 46.484375 68.546875 \r\n",
       "Q 53.21875 62.890625 53.21875 53.421875 \r\n",
       "Q 53.21875 48.921875 51.53125 44.890625 \r\n",
       "Q 49.859375 40.875 45.40625 35.40625 \r\n",
       "Q 44.1875 33.984375 37.640625 27.21875 \r\n",
       "Q 31.109375 20.453125 19.1875 8.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-50\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(66.370526 239.238437)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_2\">\r\n",
       "     <g id=\"line2d_2\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"117.284343\" xlink:href=\"#m54e93e67b9\" y=\"224.64\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_2\">\r\n",
       "      <!-- −1 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 12.40625 8.296875 \r\n",
       "L 28.515625 8.296875 \r\n",
       "L 28.515625 63.921875 \r\n",
       "L 10.984375 60.40625 \r\n",
       "L 10.984375 69.390625 \r\n",
       "L 28.421875 72.90625 \r\n",
       "L 38.28125 72.90625 \r\n",
       "L 38.28125 8.296875 \r\n",
       "L 54.390625 8.296875 \r\n",
       "L 54.390625 0 \r\n",
       "L 12.40625 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-49\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(109.913249 239.238437)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_3\">\r\n",
       "     <g id=\"line2d_3\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"160.827066\" xlink:href=\"#m54e93e67b9\" y=\"224.64\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_3\">\r\n",
       "      <!-- 0 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 31.78125 66.40625 \r\n",
       "Q 24.171875 66.40625 20.328125 58.90625 \r\n",
       "Q 16.5 51.421875 16.5 36.375 \r\n",
       "Q 16.5 21.390625 20.328125 13.890625 \r\n",
       "Q 24.171875 6.390625 31.78125 6.390625 \r\n",
       "Q 39.453125 6.390625 43.28125 13.890625 \r\n",
       "Q 47.125 21.390625 47.125 36.375 \r\n",
       "Q 47.125 51.421875 43.28125 58.90625 \r\n",
       "Q 39.453125 66.40625 31.78125 66.40625 \r\n",
       "z\r\n",
       "M 31.78125 74.21875 \r\n",
       "Q 44.046875 74.21875 50.515625 64.515625 \r\n",
       "Q 56.984375 54.828125 56.984375 36.375 \r\n",
       "Q 56.984375 17.96875 50.515625 8.265625 \r\n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \r\n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \r\n",
       "Q 6.59375 17.96875 6.59375 36.375 \r\n",
       "Q 6.59375 54.828125 13.0625 64.515625 \r\n",
       "Q 19.53125 74.21875 31.78125 74.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-48\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(157.645816 239.238437)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_4\">\r\n",
       "     <g id=\"line2d_4\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"204.369789\" xlink:href=\"#m54e93e67b9\" y=\"224.64\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_4\">\r\n",
       "      <!-- 1 -->\r\n",
       "      <g transform=\"translate(201.188539 239.238437)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_5\">\r\n",
       "     <g id=\"line2d_5\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"247.912513\" xlink:href=\"#m54e93e67b9\" y=\"224.64\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_5\">\r\n",
       "      <!-- 2 -->\r\n",
       "      <g transform=\"translate(244.731263 239.238437)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_6\">\r\n",
       "     <g id=\"line2d_6\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"291.455236\" xlink:href=\"#m54e93e67b9\" y=\"224.64\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_6\">\r\n",
       "      <!-- 3 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 40.578125 39.3125 \r\n",
       "Q 47.65625 37.796875 51.625 33 \r\n",
       "Q 55.609375 28.21875 55.609375 21.1875 \r\n",
       "Q 55.609375 10.40625 48.1875 4.484375 \r\n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \r\n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \r\n",
       "Q 12.796875 0.390625 7.625 2.203125 \r\n",
       "L 7.625 11.71875 \r\n",
       "Q 11.71875 9.328125 16.59375 8.109375 \r\n",
       "Q 21.484375 6.890625 26.8125 6.890625 \r\n",
       "Q 36.078125 6.890625 40.9375 10.546875 \r\n",
       "Q 45.796875 14.203125 45.796875 21.1875 \r\n",
       "Q 45.796875 27.640625 41.28125 31.265625 \r\n",
       "Q 36.765625 34.90625 28.71875 34.90625 \r\n",
       "L 20.21875 34.90625 \r\n",
       "L 20.21875 43.015625 \r\n",
       "L 29.109375 43.015625 \r\n",
       "Q 36.375 43.015625 40.234375 45.921875 \r\n",
       "Q 44.09375 48.828125 44.09375 54.296875 \r\n",
       "Q 44.09375 59.90625 40.109375 62.90625 \r\n",
       "Q 36.140625 65.921875 28.71875 65.921875 \r\n",
       "Q 24.65625 65.921875 20.015625 65.03125 \r\n",
       "Q 15.375 64.15625 9.8125 62.3125 \r\n",
       "L 9.8125 71.09375 \r\n",
       "Q 15.4375 72.65625 20.34375 73.4375 \r\n",
       "Q 25.25 74.21875 29.59375 74.21875 \r\n",
       "Q 40.828125 74.21875 47.359375 69.109375 \r\n",
       "Q 53.90625 64.015625 53.90625 55.328125 \r\n",
       "Q 53.90625 49.265625 50.4375 45.09375 \r\n",
       "Q 46.96875 40.921875 40.578125 39.3125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-51\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(288.273986 239.238437)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-51\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_7\">\r\n",
       "     <g id=\"line2d_7\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"334.99796\" xlink:href=\"#m54e93e67b9\" y=\"224.64\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_7\">\r\n",
       "      <!-- 4 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 37.796875 64.3125 \r\n",
       "L 12.890625 25.390625 \r\n",
       "L 37.796875 25.390625 \r\n",
       "z\r\n",
       "M 35.203125 72.90625 \r\n",
       "L 47.609375 72.90625 \r\n",
       "L 47.609375 25.390625 \r\n",
       "L 58.015625 25.390625 \r\n",
       "L 58.015625 17.1875 \r\n",
       "L 47.609375 17.1875 \r\n",
       "L 47.609375 0 \r\n",
       "L 37.796875 0 \r\n",
       "L 37.796875 17.1875 \r\n",
       "L 4.890625 17.1875 \r\n",
       "L 4.890625 26.703125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-52\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(331.81671 239.238437)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-52\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_8\">\r\n",
       "     <g id=\"line2d_8\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"378.540683\" xlink:href=\"#m54e93e67b9\" y=\"224.64\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_8\">\r\n",
       "      <!-- 5 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.796875 72.90625 \r\n",
       "L 49.515625 72.90625 \r\n",
       "L 49.515625 64.59375 \r\n",
       "L 19.828125 64.59375 \r\n",
       "L 19.828125 46.734375 \r\n",
       "Q 21.96875 47.46875 24.109375 47.828125 \r\n",
       "Q 26.265625 48.1875 28.421875 48.1875 \r\n",
       "Q 40.625 48.1875 47.75 41.5 \r\n",
       "Q 54.890625 34.8125 54.890625 23.390625 \r\n",
       "Q 54.890625 11.625 47.5625 5.09375 \r\n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \r\n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \r\n",
       "Q 12.796875 0.140625 7.71875 1.703125 \r\n",
       "L 7.71875 11.625 \r\n",
       "Q 12.109375 9.234375 16.796875 8.0625 \r\n",
       "Q 21.484375 6.890625 26.703125 6.890625 \r\n",
       "Q 35.15625 6.890625 40.078125 11.328125 \r\n",
       "Q 45.015625 15.765625 45.015625 23.390625 \r\n",
       "Q 45.015625 31 40.078125 35.4375 \r\n",
       "Q 35.15625 39.890625 26.703125 39.890625 \r\n",
       "Q 22.75 39.890625 18.8125 39.015625 \r\n",
       "Q 14.890625 38.140625 10.796875 36.28125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-53\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(375.359433 239.238437)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_9\">\r\n",
       "     <!-- x -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 54.890625 54.6875 \r\n",
       "L 35.109375 28.078125 \r\n",
       "L 55.90625 0 \r\n",
       "L 45.3125 0 \r\n",
       "L 29.390625 21.484375 \r\n",
       "L 13.484375 0 \r\n",
       "L 2.875 0 \r\n",
       "L 24.125 28.609375 \r\n",
       "L 4.6875 54.6875 \r\n",
       "L 15.28125 54.6875 \r\n",
       "L 29.78125 35.203125 \r\n",
       "L 44.28125 54.6875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-120\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(222.964063 252.916562)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_2\">\r\n",
       "    <g id=\"ytick_1\">\r\n",
       "     <g id=\"line2d_9\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L -3.5 0 \r\n",
       "\" id=\"m7889370f1a\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"218.377037\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_10\">\r\n",
       "      <!-- −1.00 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.6875 12.40625 \r\n",
       "L 21 12.40625 \r\n",
       "L 21 0 \r\n",
       "L 10.6875 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-46\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(20.878125 222.176255)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"242.822266\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_2\">\r\n",
       "     <g id=\"line2d_10\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"193.214196\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_11\">\r\n",
       "      <!-- −0.75 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 8.203125 72.90625 \r\n",
       "L 55.078125 72.90625 \r\n",
       "L 55.078125 68.703125 \r\n",
       "L 28.609375 0 \r\n",
       "L 18.3125 0 \r\n",
       "L 43.21875 64.59375 \r\n",
       "L 8.203125 64.59375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-55\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(20.878125 197.013415)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-55\"/>\r\n",
       "       <use x=\"242.822266\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_3\">\r\n",
       "     <g id=\"line2d_11\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"168.051356\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_12\">\r\n",
       "      <!-- −0.50 -->\r\n",
       "      <g transform=\"translate(20.878125 171.850575)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "       <use x=\"242.822266\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_4\">\r\n",
       "     <g id=\"line2d_12\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"142.888515\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_13\">\r\n",
       "      <!-- −0.25 -->\r\n",
       "      <g transform=\"translate(20.878125 146.687734)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "       <use x=\"242.822266\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_5\">\r\n",
       "     <g id=\"line2d_13\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"117.725675\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_14\">\r\n",
       "      <!-- 0.00 -->\r\n",
       "      <g transform=\"translate(29.257813 121.524894)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_6\">\r\n",
       "     <g id=\"line2d_14\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"92.562834\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_15\">\r\n",
       "      <!-- 0.25 -->\r\n",
       "      <g transform=\"translate(29.257813 96.362053)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_7\">\r\n",
       "     <g id=\"line2d_15\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"67.399994\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_16\">\r\n",
       "      <!-- 0.50 -->\r\n",
       "      <g transform=\"translate(29.257813 71.199213)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_8\">\r\n",
       "     <g id=\"line2d_16\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"42.237153\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_17\">\r\n",
       "      <!-- 0.75 -->\r\n",
       "      <g transform=\"translate(29.257813 46.036372)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-55\"/>\r\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_9\">\r\n",
       "     <g id=\"line2d_17\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"58.523438\" xlink:href=\"#m7889370f1a\" y=\"17.074313\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_18\">\r\n",
       "      <!-- 1.00 -->\r\n",
       "      <g transform=\"translate(29.257813 20.873532)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"159.033203\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"text_19\">\r\n",
       "     <!-- f(x) -->\r\n",
       "     <defs>\r\n",
       "      <path d=\"M 37.109375 75.984375 \r\n",
       "L 37.109375 68.5 \r\n",
       "L 28.515625 68.5 \r\n",
       "Q 23.6875 68.5 21.796875 66.546875 \r\n",
       "Q 19.921875 64.59375 19.921875 59.515625 \r\n",
       "L 19.921875 54.6875 \r\n",
       "L 34.71875 54.6875 \r\n",
       "L 34.71875 47.703125 \r\n",
       "L 19.921875 47.703125 \r\n",
       "L 19.921875 0 \r\n",
       "L 10.890625 0 \r\n",
       "L 10.890625 47.703125 \r\n",
       "L 2.296875 47.703125 \r\n",
       "L 2.296875 54.6875 \r\n",
       "L 10.890625 54.6875 \r\n",
       "L 10.890625 58.5 \r\n",
       "Q 10.890625 67.625 15.140625 71.796875 \r\n",
       "Q 19.390625 75.984375 28.609375 75.984375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-102\"/>\r\n",
       "      <path d=\"M 31 75.875 \r\n",
       "Q 24.46875 64.65625 21.28125 53.65625 \r\n",
       "Q 18.109375 42.671875 18.109375 31.390625 \r\n",
       "Q 18.109375 20.125 21.3125 9.0625 \r\n",
       "Q 24.515625 -2 31 -13.1875 \r\n",
       "L 23.1875 -13.1875 \r\n",
       "Q 15.875 -1.703125 12.234375 9.375 \r\n",
       "Q 8.59375 20.453125 8.59375 31.390625 \r\n",
       "Q 8.59375 42.28125 12.203125 53.3125 \r\n",
       "Q 15.828125 64.359375 23.1875 75.875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-40\"/>\r\n",
       "      <path d=\"M 8.015625 75.875 \r\n",
       "L 15.828125 75.875 \r\n",
       "Q 23.140625 64.359375 26.78125 53.3125 \r\n",
       "Q 30.421875 42.28125 30.421875 31.390625 \r\n",
       "Q 30.421875 20.453125 26.78125 9.375 \r\n",
       "Q 23.140625 -1.703125 15.828125 -13.1875 \r\n",
       "L 8.015625 -13.1875 \r\n",
       "Q 14.5 -2 17.703125 9.0625 \r\n",
       "Q 20.90625 20.125 20.90625 31.390625 \r\n",
       "Q 20.90625 42.671875 17.703125 53.65625 \r\n",
       "Q 14.5 64.65625 8.015625 75.875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-41\"/>\r\n",
       "     </defs>\r\n",
       "     <g transform=\"translate(14.798438 124.541094)rotate(-90)scale(0.1 -0.1)\">\r\n",
       "      <use xlink:href=\"#DejaVuSans-102\"/>\r\n",
       "      <use x=\"35.205078\" xlink:href=\"#DejaVuSans-40\"/>\r\n",
       "      <use x=\"74.21875\" xlink:href=\"#DejaVuSans-120\"/>\r\n",
       "      <use x=\"133.398438\" xlink:href=\"#DejaVuSans-41\"/>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_18\">\r\n",
       "    <path clip-path=\"url(#pee6ce2047d)\" d=\"M 73.741619 214.756364 \r\n",
       "L 78.966746 213.796606 \r\n",
       "L 83.756446 212.701248 \r\n",
       "L 88.110718 211.487621 \r\n",
       "L 92.029563 210.184206 \r\n",
       "L 95.948408 208.64654 \r\n",
       "L 99.431826 207.053168 \r\n",
       "L 102.915244 205.216804 \r\n",
       "L 106.398662 203.106578 \r\n",
       "L 109.446652 201.009806 \r\n",
       "L 112.494643 198.65565 \r\n",
       "L 115.542634 196.021168 \r\n",
       "L 118.590624 193.083781 \r\n",
       "L 121.638615 189.822031 \r\n",
       "L 124.686606 186.216512 \r\n",
       "L 127.734596 182.250944 \r\n",
       "L 130.782587 177.913378 \r\n",
       "L 133.830578 173.197461 \r\n",
       "L 136.878568 168.103716 \r\n",
       "L 140.361986 161.831033 \r\n",
       "L 143.845404 155.103588 \r\n",
       "L 147.764249 147.046686 \r\n",
       "L 152.118521 137.59177 \r\n",
       "L 158.214503 123.75752 \r\n",
       "L 170.406465 95.932837 \r\n",
       "L 175.196165 85.666125 \r\n",
       "L 179.11501 77.774086 \r\n",
       "L 182.598428 71.212954 \r\n",
       "L 186.081846 65.118688 \r\n",
       "L 189.565263 59.51261 \r\n",
       "L 192.613254 55.013299 \r\n",
       "L 195.661245 50.88947 \r\n",
       "L 198.709235 47.131413 \r\n",
       "L 201.757226 43.724554 \r\n",
       "L 204.805217 40.650685 \r\n",
       "L 207.853207 37.889106 \r\n",
       "L 210.901198 35.417613 \r\n",
       "L 213.949189 33.213339 \r\n",
       "L 217.432606 30.992065 \r\n",
       "L 220.916024 29.05673 \r\n",
       "L 224.399442 27.375718 \r\n",
       "L 228.318287 25.751921 \r\n",
       "L 232.237132 24.374298 \r\n",
       "L 236.591405 23.090583 \r\n",
       "L 241.381104 21.931148 \r\n",
       "L 246.606231 20.914605 \r\n",
       "L 252.266785 20.048366 \r\n",
       "L 258.798194 19.286015 \r\n",
       "L 266.200457 18.653538 \r\n",
       "L 274.909001 18.135644 \r\n",
       "L 285.794682 17.719381 \r\n",
       "L 300.163781 17.408234 \r\n",
       "L 321.064288 17.2023 \r\n",
       "L 358.075603 17.097706 \r\n",
       "L 378.105256 17.083636 \r\n",
       "L 378.105256 17.083636 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_3\">\r\n",
       "    <path d=\"M 58.523438 224.64 \r\n",
       "L 58.523438 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_4\">\r\n",
       "    <path d=\"M 393.323438 224.64 \r\n",
       "L 393.323438 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_5\">\r\n",
       "    <path d=\"M 58.523438 224.64 \r\n",
       "L 393.323438 224.64 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_6\">\r\n",
       "    <path d=\"M 58.523438 7.2 \r\n",
       "L 393.323438 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_7\">\r\n",
       "    <path d=\"M 298.23085 106.280219 \r\n",
       "Q 316.233471 62.601503 333.810052 19.956465 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "    <path d=\"M 330.436704 22.892538 \r\n",
       "L 333.810052 19.956465 \r\n",
       "L 334.134901 24.416787 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:round;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"text_20\">\r\n",
       "    <!-- vanishing gradient -->\r\n",
       "    <defs>\r\n",
       "     <path d=\"M 2.984375 54.6875 \r\n",
       "L 12.5 54.6875 \r\n",
       "L 29.59375 8.796875 \r\n",
       "L 46.6875 54.6875 \r\n",
       "L 56.203125 54.6875 \r\n",
       "L 35.6875 0 \r\n",
       "L 23.484375 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-118\"/>\r\n",
       "     <path d=\"M 34.28125 27.484375 \r\n",
       "Q 23.390625 27.484375 19.1875 25 \r\n",
       "Q 14.984375 22.515625 14.984375 16.5 \r\n",
       "Q 14.984375 11.71875 18.140625 8.90625 \r\n",
       "Q 21.296875 6.109375 26.703125 6.109375 \r\n",
       "Q 34.1875 6.109375 38.703125 11.40625 \r\n",
       "Q 43.21875 16.703125 43.21875 25.484375 \r\n",
       "L 43.21875 27.484375 \r\n",
       "z\r\n",
       "M 52.203125 31.203125 \r\n",
       "L 52.203125 0 \r\n",
       "L 43.21875 0 \r\n",
       "L 43.21875 8.296875 \r\n",
       "Q 40.140625 3.328125 35.546875 0.953125 \r\n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \r\n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \r\n",
       "Q 6 8.015625 6 15.921875 \r\n",
       "Q 6 25.140625 12.171875 29.828125 \r\n",
       "Q 18.359375 34.515625 30.609375 34.515625 \r\n",
       "L 43.21875 34.515625 \r\n",
       "L 43.21875 35.40625 \r\n",
       "Q 43.21875 41.609375 39.140625 45 \r\n",
       "Q 35.0625 48.390625 27.6875 48.390625 \r\n",
       "Q 23 48.390625 18.546875 47.265625 \r\n",
       "Q 14.109375 46.140625 10.015625 43.890625 \r\n",
       "L 10.015625 52.203125 \r\n",
       "Q 14.9375 54.109375 19.578125 55.046875 \r\n",
       "Q 24.21875 56 28.609375 56 \r\n",
       "Q 40.484375 56 46.34375 49.84375 \r\n",
       "Q 52.203125 43.703125 52.203125 31.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-97\"/>\r\n",
       "     <path d=\"M 54.890625 33.015625 \r\n",
       "L 54.890625 0 \r\n",
       "L 45.90625 0 \r\n",
       "L 45.90625 32.71875 \r\n",
       "Q 45.90625 40.484375 42.875 44.328125 \r\n",
       "Q 39.84375 48.1875 33.796875 48.1875 \r\n",
       "Q 26.515625 48.1875 22.3125 43.546875 \r\n",
       "Q 18.109375 38.921875 18.109375 30.90625 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 21.34375 51.125 25.703125 53.5625 \r\n",
       "Q 30.078125 56 35.796875 56 \r\n",
       "Q 45.21875 56 50.046875 50.171875 \r\n",
       "Q 54.890625 44.34375 54.890625 33.015625 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-110\"/>\r\n",
       "     <path d=\"M 9.421875 54.6875 \r\n",
       "L 18.40625 54.6875 \r\n",
       "L 18.40625 0 \r\n",
       "L 9.421875 0 \r\n",
       "z\r\n",
       "M 9.421875 75.984375 \r\n",
       "L 18.40625 75.984375 \r\n",
       "L 18.40625 64.59375 \r\n",
       "L 9.421875 64.59375 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-105\"/>\r\n",
       "     <path d=\"M 44.28125 53.078125 \r\n",
       "L 44.28125 44.578125 \r\n",
       "Q 40.484375 46.53125 36.375 47.5 \r\n",
       "Q 32.28125 48.484375 27.875 48.484375 \r\n",
       "Q 21.1875 48.484375 17.84375 46.4375 \r\n",
       "Q 14.5 44.390625 14.5 40.28125 \r\n",
       "Q 14.5 37.15625 16.890625 35.375 \r\n",
       "Q 19.28125 33.59375 26.515625 31.984375 \r\n",
       "L 29.59375 31.296875 \r\n",
       "Q 39.15625 29.25 43.1875 25.515625 \r\n",
       "Q 47.21875 21.78125 47.21875 15.09375 \r\n",
       "Q 47.21875 7.46875 41.1875 3.015625 \r\n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \r\n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \r\n",
       "Q 10.6875 0.296875 5.421875 2 \r\n",
       "L 5.421875 11.28125 \r\n",
       "Q 10.40625 8.6875 15.234375 7.390625 \r\n",
       "Q 20.0625 6.109375 24.8125 6.109375 \r\n",
       "Q 31.15625 6.109375 34.5625 8.28125 \r\n",
       "Q 37.984375 10.453125 37.984375 14.40625 \r\n",
       "Q 37.984375 18.0625 35.515625 20.015625 \r\n",
       "Q 33.0625 21.96875 24.703125 23.78125 \r\n",
       "L 21.578125 24.515625 \r\n",
       "Q 13.234375 26.265625 9.515625 29.90625 \r\n",
       "Q 5.8125 33.546875 5.8125 39.890625 \r\n",
       "Q 5.8125 47.609375 11.28125 51.796875 \r\n",
       "Q 16.75 56 26.8125 56 \r\n",
       "Q 31.78125 56 36.171875 55.265625 \r\n",
       "Q 40.578125 54.546875 44.28125 53.078125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-115\"/>\r\n",
       "     <path d=\"M 54.890625 33.015625 \r\n",
       "L 54.890625 0 \r\n",
       "L 45.90625 0 \r\n",
       "L 45.90625 32.71875 \r\n",
       "Q 45.90625 40.484375 42.875 44.328125 \r\n",
       "Q 39.84375 48.1875 33.796875 48.1875 \r\n",
       "Q 26.515625 48.1875 22.3125 43.546875 \r\n",
       "Q 18.109375 38.921875 18.109375 30.90625 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 75.984375 \r\n",
       "L 18.109375 75.984375 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 21.34375 51.125 25.703125 53.5625 \r\n",
       "Q 30.078125 56 35.796875 56 \r\n",
       "Q 45.21875 56 50.046875 50.171875 \r\n",
       "Q 54.890625 44.34375 54.890625 33.015625 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-104\"/>\r\n",
       "     <path d=\"M 45.40625 27.984375 \r\n",
       "Q 45.40625 37.75 41.375 43.109375 \r\n",
       "Q 37.359375 48.484375 30.078125 48.484375 \r\n",
       "Q 22.859375 48.484375 18.828125 43.109375 \r\n",
       "Q 14.796875 37.75 14.796875 27.984375 \r\n",
       "Q 14.796875 18.265625 18.828125 12.890625 \r\n",
       "Q 22.859375 7.515625 30.078125 7.515625 \r\n",
       "Q 37.359375 7.515625 41.375 12.890625 \r\n",
       "Q 45.40625 18.265625 45.40625 27.984375 \r\n",
       "z\r\n",
       "M 54.390625 6.78125 \r\n",
       "Q 54.390625 -7.171875 48.1875 -13.984375 \r\n",
       "Q 42 -20.796875 29.203125 -20.796875 \r\n",
       "Q 24.46875 -20.796875 20.265625 -20.09375 \r\n",
       "Q 16.0625 -19.390625 12.109375 -17.921875 \r\n",
       "L 12.109375 -9.1875 \r\n",
       "Q 16.0625 -11.328125 19.921875 -12.34375 \r\n",
       "Q 23.78125 -13.375 27.78125 -13.375 \r\n",
       "Q 36.625 -13.375 41.015625 -8.765625 \r\n",
       "Q 45.40625 -4.15625 45.40625 5.171875 \r\n",
       "L 45.40625 9.625 \r\n",
       "Q 42.625 4.78125 38.28125 2.390625 \r\n",
       "Q 33.9375 0 27.875 0 \r\n",
       "Q 17.828125 0 11.671875 7.65625 \r\n",
       "Q 5.515625 15.328125 5.515625 27.984375 \r\n",
       "Q 5.515625 40.671875 11.671875 48.328125 \r\n",
       "Q 17.828125 56 27.875 56 \r\n",
       "Q 33.9375 56 38.28125 53.609375 \r\n",
       "Q 42.625 51.21875 45.40625 46.390625 \r\n",
       "L 45.40625 54.6875 \r\n",
       "L 54.390625 54.6875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-103\"/>\r\n",
       "     <path id=\"DejaVuSans-32\"/>\r\n",
       "     <path d=\"M 41.109375 46.296875 \r\n",
       "Q 39.59375 47.171875 37.8125 47.578125 \r\n",
       "Q 36.03125 48 33.890625 48 \r\n",
       "Q 26.265625 48 22.1875 43.046875 \r\n",
       "Q 18.109375 38.09375 18.109375 28.8125 \r\n",
       "L 18.109375 0 \r\n",
       "L 9.078125 0 \r\n",
       "L 9.078125 54.6875 \r\n",
       "L 18.109375 54.6875 \r\n",
       "L 18.109375 46.1875 \r\n",
       "Q 20.953125 51.171875 25.484375 53.578125 \r\n",
       "Q 30.03125 56 36.53125 56 \r\n",
       "Q 37.453125 56 38.578125 55.875 \r\n",
       "Q 39.703125 55.765625 41.0625 55.515625 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-114\"/>\r\n",
       "     <path d=\"M 45.40625 46.390625 \r\n",
       "L 45.40625 75.984375 \r\n",
       "L 54.390625 75.984375 \r\n",
       "L 54.390625 0 \r\n",
       "L 45.40625 0 \r\n",
       "L 45.40625 8.203125 \r\n",
       "Q 42.578125 3.328125 38.25 0.953125 \r\n",
       "Q 33.9375 -1.421875 27.875 -1.421875 \r\n",
       "Q 17.96875 -1.421875 11.734375 6.484375 \r\n",
       "Q 5.515625 14.40625 5.515625 27.296875 \r\n",
       "Q 5.515625 40.1875 11.734375 48.09375 \r\n",
       "Q 17.96875 56 27.875 56 \r\n",
       "Q 33.9375 56 38.25 53.625 \r\n",
       "Q 42.578125 51.265625 45.40625 46.390625 \r\n",
       "z\r\n",
       "M 14.796875 27.296875 \r\n",
       "Q 14.796875 17.390625 18.875 11.75 \r\n",
       "Q 22.953125 6.109375 30.078125 6.109375 \r\n",
       "Q 37.203125 6.109375 41.296875 11.75 \r\n",
       "Q 45.40625 17.390625 45.40625 27.296875 \r\n",
       "Q 45.40625 37.203125 41.296875 42.84375 \r\n",
       "Q 37.203125 48.484375 30.078125 48.484375 \r\n",
       "Q 22.953125 48.484375 18.875 42.84375 \r\n",
       "Q 14.796875 37.203125 14.796875 27.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-100\"/>\r\n",
       "     <path d=\"M 56.203125 29.59375 \r\n",
       "L 56.203125 25.203125 \r\n",
       "L 14.890625 25.203125 \r\n",
       "Q 15.484375 15.921875 20.484375 11.0625 \r\n",
       "Q 25.484375 6.203125 34.421875 6.203125 \r\n",
       "Q 39.59375 6.203125 44.453125 7.46875 \r\n",
       "Q 49.3125 8.734375 54.109375 11.28125 \r\n",
       "L 54.109375 2.78125 \r\n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \r\n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \r\n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \r\n",
       "Q 5.515625 13.8125 5.515625 26.8125 \r\n",
       "Q 5.515625 40.234375 12.765625 48.109375 \r\n",
       "Q 20.015625 56 32.328125 56 \r\n",
       "Q 43.359375 56 49.78125 48.890625 \r\n",
       "Q 56.203125 41.796875 56.203125 29.59375 \r\n",
       "z\r\n",
       "M 47.21875 32.234375 \r\n",
       "Q 47.125 39.59375 43.09375 43.984375 \r\n",
       "Q 39.0625 48.390625 32.421875 48.390625 \r\n",
       "Q 24.90625 48.390625 20.390625 44.140625 \r\n",
       "Q 15.875 39.890625 15.1875 32.171875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-101\"/>\r\n",
       "     <path d=\"M 18.3125 70.21875 \r\n",
       "L 18.3125 54.6875 \r\n",
       "L 36.8125 54.6875 \r\n",
       "L 36.8125 47.703125 \r\n",
       "L 18.3125 47.703125 \r\n",
       "L 18.3125 18.015625 \r\n",
       "Q 18.3125 11.328125 20.140625 9.421875 \r\n",
       "Q 21.96875 7.515625 27.59375 7.515625 \r\n",
       "L 36.8125 7.515625 \r\n",
       "L 36.8125 0 \r\n",
       "L 27.59375 0 \r\n",
       "Q 17.1875 0 13.234375 3.875 \r\n",
       "Q 9.28125 7.765625 9.28125 18.015625 \r\n",
       "L 9.28125 47.703125 \r\n",
       "L 2.6875 47.703125 \r\n",
       "L 2.6875 54.6875 \r\n",
       "L 9.28125 54.6875 \r\n",
       "L 9.28125 70.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-116\"/>\r\n",
       "    </defs>\r\n",
       "    <g transform=\"translate(247.912513 117.725675)scale(0.1 -0.1)\">\r\n",
       "     <use xlink:href=\"#DejaVuSans-118\"/>\r\n",
       "     <use x=\"59.179688\" xlink:href=\"#DejaVuSans-97\"/>\r\n",
       "     <use x=\"120.458984\" xlink:href=\"#DejaVuSans-110\"/>\r\n",
       "     <use x=\"183.837891\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"211.621094\" xlink:href=\"#DejaVuSans-115\"/>\r\n",
       "     <use x=\"263.720703\" xlink:href=\"#DejaVuSans-104\"/>\r\n",
       "     <use x=\"327.099609\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"354.882812\" xlink:href=\"#DejaVuSans-110\"/>\r\n",
       "     <use x=\"418.261719\" xlink:href=\"#DejaVuSans-103\"/>\r\n",
       "     <use x=\"481.738281\" xlink:href=\"#DejaVuSans-32\"/>\r\n",
       "     <use x=\"513.525391\" xlink:href=\"#DejaVuSans-103\"/>\r\n",
       "     <use x=\"577.001953\" xlink:href=\"#DejaVuSans-114\"/>\r\n",
       "     <use x=\"618.115234\" xlink:href=\"#DejaVuSans-97\"/>\r\n",
       "     <use x=\"679.394531\" xlink:href=\"#DejaVuSans-100\"/>\r\n",
       "     <use x=\"742.871094\" xlink:href=\"#DejaVuSans-105\"/>\r\n",
       "     <use x=\"770.654297\" xlink:href=\"#DejaVuSans-101\"/>\r\n",
       "     <use x=\"832.177734\" xlink:href=\"#DejaVuSans-110\"/>\r\n",
       "     <use x=\"895.556641\" xlink:href=\"#DejaVuSans-116\"/>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       " </g>\r\n",
       " <defs>\r\n",
       "  <clipPath id=\"pee6ce2047d\">\r\n",
       "   <rect height=\"217.44\" width=\"334.8\" x=\"58.523438\" y=\"7.2\"/>\r\n",
       "  </clipPath>\r\n",
       " </defs>\r\n",
       "</svg>\r\n"
      ],
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.arange(-2.0, 5.0, 0.01)\n",
    "fig, = d2l.plt.plot(x, np.tanh(x))\n",
    "d2l.plt.xlabel('x')\n",
    "d2l.plt.ylabel('f(x)')\n",
    "fig.axes.annotate('vanishing gradient', (4, 1), (2, 0.0) ,arrowprops=dict(arrowstyle='->'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_bziy6ld",
    "id": "41A4D17DC7054EAD9201E44FE548EFEF",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# 凸性 （Convexity）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_vbicm9n",
    "id": "BCF4D1F31ECE493F83A331FF37F97F33",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 基础"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_pyk6zer",
    "id": "C2029EA08486473D8458254669DF04FA",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 集合\n",
    "凸集合的交集必定是凸集合，凸集合的并集不一定是凸集合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_vsxc7cx",
    "id": "246D957762404B9FA76FC3392B96BBF1",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "![Image Name](https://cdn.kesci.com/upload/image/q5p1yhqzm8.png?imageView2/0/w/640/h/640)\n",
    "![Image Name](https://cdn.kesci.com/upload/image/q5p1xz9gvz.png?imageView2/0/w/640/h/640)\n",
    "![Image Name](https://cdn.kesci.com/upload/image/q5p1yue9fu.png?imageView2/0/w/320/h/320)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_6bgwsca",
    "id": "4363326866B143D2824388129682629D",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 函数\n",
    "下式表示函数值的期望大于期望的函数值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_ulkhral",
    "id": "E2DAB2658B334DFA8FEDB2D714B79A64",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "\n",
    "$$\n",
    "\\lambda f(x)+(1-\\lambda) f\\left(x^{\\prime}\\right) \\geq f\\left(\\lambda x+(1-\\lambda) x^{\\prime}\\right)\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "graffitiCellId": "id_1uvaa45",
    "id": "507C2126C2654EAC8A2C220434232A3F",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\r\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\r\n",
       "<svg height=\"194.158125pt\" version=\"1.1\" viewBox=\"0 0 539.503125 194.158125\" width=\"539.503125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n",
       " <defs>\r\n",
       "  <style type=\"text/css\">\r\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;}\r\n",
       "  </style>\r\n",
       " </defs>\r\n",
       " <g id=\"figure_1\">\r\n",
       "  <g id=\"patch_1\">\r\n",
       "   <path d=\"M 0 194.158125 \r\n",
       "L 539.503125 194.158125 \r\n",
       "L 539.503125 0 \r\n",
       "L 0 0 \r\n",
       "z\r\n",
       "\" style=\"fill:none;\"/>\r\n",
       "  </g>\r\n",
       "  <g id=\"axes_1\">\r\n",
       "   <g id=\"patch_2\">\r\n",
       "    <path d=\"M 30.103125 170.28 \r\n",
       "L 177.809007 170.28 \r\n",
       "L 177.809007 7.2 \r\n",
       "L 30.103125 7.2 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_1\">\r\n",
       "    <g id=\"xtick_1\">\r\n",
       "     <g id=\"line2d_1\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L 0 3.5 \r\n",
       "\" id=\"m9899e5487b\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"36.817029\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_1\">\r\n",
       "      <!-- −2 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.59375 35.5 \r\n",
       "L 73.1875 35.5 \r\n",
       "L 73.1875 27.203125 \r\n",
       "L 10.59375 27.203125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-8722\"/>\r\n",
       "       <path d=\"M 19.1875 8.296875 \r\n",
       "L 53.609375 8.296875 \r\n",
       "L 53.609375 0 \r\n",
       "L 7.328125 0 \r\n",
       "L 7.328125 8.296875 \r\n",
       "Q 12.9375 14.109375 22.625 23.890625 \r\n",
       "Q 32.328125 33.6875 34.8125 36.53125 \r\n",
       "Q 39.546875 41.84375 41.421875 45.53125 \r\n",
       "Q 43.3125 49.21875 43.3125 52.78125 \r\n",
       "Q 43.3125 58.59375 39.234375 62.25 \r\n",
       "Q 35.15625 65.921875 28.609375 65.921875 \r\n",
       "Q 23.96875 65.921875 18.8125 64.3125 \r\n",
       "Q 13.671875 62.703125 7.8125 59.421875 \r\n",
       "L 7.8125 69.390625 \r\n",
       "Q 13.765625 71.78125 18.9375 73 \r\n",
       "Q 24.125 74.21875 28.421875 74.21875 \r\n",
       "Q 39.75 74.21875 46.484375 68.546875 \r\n",
       "Q 53.21875 62.890625 53.21875 53.421875 \r\n",
       "Q 53.21875 48.921875 51.53125 44.890625 \r\n",
       "Q 49.859375 40.875 45.40625 35.40625 \r\n",
       "Q 44.1875 33.984375 37.640625 27.21875 \r\n",
       "Q 31.109375 20.453125 19.1875 8.296875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-50\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(29.445935 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_2\">\r\n",
       "     <g id=\"line2d_2\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"104.124334\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_2\">\r\n",
       "      <!-- 0 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 31.78125 66.40625 \r\n",
       "Q 24.171875 66.40625 20.328125 58.90625 \r\n",
       "Q 16.5 51.421875 16.5 36.375 \r\n",
       "Q 16.5 21.390625 20.328125 13.890625 \r\n",
       "Q 24.171875 6.390625 31.78125 6.390625 \r\n",
       "Q 39.453125 6.390625 43.28125 13.890625 \r\n",
       "Q 47.125 21.390625 47.125 36.375 \r\n",
       "Q 47.125 51.421875 43.28125 58.90625 \r\n",
       "Q 39.453125 66.40625 31.78125 66.40625 \r\n",
       "z\r\n",
       "M 31.78125 74.21875 \r\n",
       "Q 44.046875 74.21875 50.515625 64.515625 \r\n",
       "Q 56.984375 54.828125 56.984375 36.375 \r\n",
       "Q 56.984375 17.96875 50.515625 8.265625 \r\n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \r\n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \r\n",
       "Q 6.59375 17.96875 6.59375 36.375 \r\n",
       "Q 6.59375 54.828125 13.0625 64.515625 \r\n",
       "Q 19.53125 74.21875 31.78125 74.21875 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-48\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(100.943084 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_3\">\r\n",
       "     <g id=\"line2d_3\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"171.43164\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_3\">\r\n",
       "      <!-- 2 -->\r\n",
       "      <g transform=\"translate(168.25039 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_2\">\r\n",
       "    <g id=\"ytick_1\">\r\n",
       "     <g id=\"line2d_4\">\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 0 0 \r\n",
       "L -3.5 0 \r\n",
       "\" id=\"m259f5d8728\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n",
       "      </defs>\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"30.103125\" xlink:href=\"#m259f5d8728\" y=\"162.867273\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_4\">\r\n",
       "      <!-- 0.0 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.6875 12.40625 \r\n",
       "L 21 12.40625 \r\n",
       "L 21 0 \r\n",
       "L 10.6875 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-46\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(7.2 166.666491)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_2\">\r\n",
       "     <g id=\"line2d_5\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"30.103125\" xlink:href=\"#m259f5d8728\" y=\"125.803636\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_5\">\r\n",
       "      <!-- 0.5 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 10.796875 72.90625 \r\n",
       "L 49.515625 72.90625 \r\n",
       "L 49.515625 64.59375 \r\n",
       "L 19.828125 64.59375 \r\n",
       "L 19.828125 46.734375 \r\n",
       "Q 21.96875 47.46875 24.109375 47.828125 \r\n",
       "Q 26.265625 48.1875 28.421875 48.1875 \r\n",
       "Q 40.625 48.1875 47.75 41.5 \r\n",
       "Q 54.890625 34.8125 54.890625 23.390625 \r\n",
       "Q 54.890625 11.625 47.5625 5.09375 \r\n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \r\n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \r\n",
       "Q 12.796875 0.140625 7.71875 1.703125 \r\n",
       "L 7.71875 11.625 \r\n",
       "Q 12.109375 9.234375 16.796875 8.0625 \r\n",
       "Q 21.484375 6.890625 26.703125 6.890625 \r\n",
       "Q 35.15625 6.890625 40.078125 11.328125 \r\n",
       "Q 45.015625 15.765625 45.015625 23.390625 \r\n",
       "Q 45.015625 31 40.078125 35.4375 \r\n",
       "Q 35.15625 39.890625 26.703125 39.890625 \r\n",
       "Q 22.75 39.890625 18.8125 39.015625 \r\n",
       "Q 14.890625 38.140625 10.796875 36.28125 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-53\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(7.2 129.602855)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_3\">\r\n",
       "     <g id=\"line2d_6\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"30.103125\" xlink:href=\"#m259f5d8728\" y=\"88.74\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_6\">\r\n",
       "      <!-- 1.0 -->\r\n",
       "      <defs>\r\n",
       "       <path d=\"M 12.40625 8.296875 \r\n",
       "L 28.515625 8.296875 \r\n",
       "L 28.515625 63.921875 \r\n",
       "L 10.984375 60.40625 \r\n",
       "L 10.984375 69.390625 \r\n",
       "L 28.421875 72.90625 \r\n",
       "L 38.28125 72.90625 \r\n",
       "L 38.28125 8.296875 \r\n",
       "L 54.390625 8.296875 \r\n",
       "L 54.390625 0 \r\n",
       "L 12.40625 0 \r\n",
       "z\r\n",
       "\" id=\"DejaVuSans-49\"/>\r\n",
       "      </defs>\r\n",
       "      <g transform=\"translate(7.2 92.539219)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_4\">\r\n",
       "     <g id=\"line2d_7\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"30.103125\" xlink:href=\"#m259f5d8728\" y=\"51.676364\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_7\">\r\n",
       "      <!-- 1.5 -->\r\n",
       "      <g transform=\"translate(7.2 55.475582)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_5\">\r\n",
       "     <g id=\"line2d_8\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"30.103125\" xlink:href=\"#m259f5d8728\" y=\"14.612727\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_8\">\r\n",
       "      <!-- 2.0 -->\r\n",
       "      <g transform=\"translate(7.2 18.411946)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_9\">\r\n",
       "    <path clip-path=\"url(#p50ba7a7102)\" d=\"M 36.817029 14.612727 \r\n",
       "L 40.855467 31.869556 \r\n",
       "L 44.557369 46.750606 \r\n",
       "L 48.259271 60.734716 \r\n",
       "L 51.961173 73.821886 \r\n",
       "L 55.326538 84.940977 \r\n",
       "L 58.691903 95.318795 \r\n",
       "L 62.057268 104.955341 \r\n",
       "L 65.086097 112.994444 \r\n",
       "L 68.114926 120.433115 \r\n",
       "L 71.143755 127.271356 \r\n",
       "L 73.836047 132.845727 \r\n",
       "L 76.528339 137.945684 \r\n",
       "L 79.220631 142.571225 \r\n",
       "L 81.912924 146.722353 \r\n",
       "L 84.268679 149.965421 \r\n",
       "L 86.624435 152.845265 \r\n",
       "L 88.980191 155.361886 \r\n",
       "L 90.99941 157.229894 \r\n",
       "L 93.018629 158.831043 \r\n",
       "L 95.037848 160.165334 \r\n",
       "L 97.057067 161.232766 \r\n",
       "L 99.076287 162.033341 \r\n",
       "L 101.095506 162.567057 \r\n",
       "L 103.114725 162.833915 \r\n",
       "L 105.133944 162.833915 \r\n",
       "L 107.153163 162.567057 \r\n",
       "L 109.172382 162.033341 \r\n",
       "L 111.191602 161.232766 \r\n",
       "L 113.210821 160.165334 \r\n",
       "L 115.23004 158.831043 \r\n",
       "L 117.249259 157.229894 \r\n",
       "L 119.268478 155.361886 \r\n",
       "L 121.624234 152.845265 \r\n",
       "L 123.97999 149.965421 \r\n",
       "L 126.335745 146.722353 \r\n",
       "L 128.691501 143.116061 \r\n",
       "L 131.383793 138.549821 \r\n",
       "L 134.076085 133.509166 \r\n",
       "L 136.768378 127.994097 \r\n",
       "L 139.46067 122.004614 \r\n",
       "L 142.489499 114.699371 \r\n",
       "L 145.518327 106.793697 \r\n",
       "L 148.547156 98.287593 \r\n",
       "L 151.912521 88.132156 \r\n",
       "L 155.277887 77.235447 \r\n",
       "L 158.643252 65.597465 \r\n",
       "L 162.345154 51.939515 \r\n",
       "L 166.047056 37.384625 \r\n",
       "L 169.748957 21.932795 \r\n",
       "L 171.095104 16.091566 \r\n",
       "L 171.095104 16.091566 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_10\">\r\n",
       "    <path clip-path=\"url(#p50ba7a7102)\" d=\"M 53.643855 79.474091 \r\n",
       "L 137.777987 125.803636 \r\n",
       "\" style=\"fill:none;stroke:#800080;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_3\">\r\n",
       "    <path d=\"M 30.103125 170.28 \r\n",
       "L 30.103125 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_4\">\r\n",
       "    <path d=\"M 177.809007 170.28 \r\n",
       "L 177.809007 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_5\">\r\n",
       "    <path d=\"M 30.103125 170.28 \r\n",
       "L 177.809007 170.28 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_6\">\r\n",
       "    <path d=\"M 30.103125 7.2 \r\n",
       "L 177.809007 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       "  <g id=\"axes_2\">\r\n",
       "   <g id=\"patch_7\">\r\n",
       "    <path d=\"M 207.350184 170.28 \r\n",
       "L 355.056066 170.28 \r\n",
       "L 355.056066 7.2 \r\n",
       "L 207.350184 7.2 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_3\">\r\n",
       "    <g id=\"xtick_4\">\r\n",
       "     <g id=\"line2d_11\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"214.064088\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_9\">\r\n",
       "      <!-- −2 -->\r\n",
       "      <g transform=\"translate(206.692994 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_5\">\r\n",
       "     <g id=\"line2d_12\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"281.371393\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_10\">\r\n",
       "      <!-- 0 -->\r\n",
       "      <g transform=\"translate(278.190143 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_6\">\r\n",
       "     <g id=\"line2d_13\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"348.678699\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_11\">\r\n",
       "      <!-- 2 -->\r\n",
       "      <g transform=\"translate(345.497449 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_4\">\r\n",
       "    <g id=\"ytick_6\">\r\n",
       "     <g id=\"line2d_14\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"207.350184\" xlink:href=\"#m259f5d8728\" y=\"162.867273\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_12\">\r\n",
       "      <!-- −1.0 -->\r\n",
       "      <g transform=\"translate(176.067371 166.666491)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_7\">\r\n",
       "     <g id=\"line2d_15\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"207.350184\" xlink:href=\"#m259f5d8728\" y=\"125.803636\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_13\">\r\n",
       "      <!-- −0.5 -->\r\n",
       "      <g transform=\"translate(176.067371 129.602855)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"147.412109\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"179.199219\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_8\">\r\n",
       "     <g id=\"line2d_16\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"207.350184\" xlink:href=\"#m259f5d8728\" y=\"88.74\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_14\">\r\n",
       "      <!-- 0.0 -->\r\n",
       "      <g transform=\"translate(184.447059 92.539219)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_9\">\r\n",
       "     <g id=\"line2d_17\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"207.350184\" xlink:href=\"#m259f5d8728\" y=\"51.676364\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_15\">\r\n",
       "      <!-- 0.5 -->\r\n",
       "      <g transform=\"translate(184.447059 55.475582)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_10\">\r\n",
       "     <g id=\"line2d_18\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"207.350184\" xlink:href=\"#m259f5d8728\" y=\"14.612727\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_16\">\r\n",
       "      <!-- 1.0 -->\r\n",
       "      <g transform=\"translate(184.447059 18.411946)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_19\">\r\n",
       "    <path clip-path=\"url(#p884e1a932c)\" d=\"M 214.064088 14.612727 \r\n",
       "L 214.737161 14.759001 \r\n",
       "L 215.410234 15.197243 \r\n",
       "L 216.083307 15.925725 \r\n",
       "L 216.75638 16.941572 \r\n",
       "L 217.765989 18.995075 \r\n",
       "L 218.775599 21.667638 \r\n",
       "L 219.785209 24.935541 \r\n",
       "L 221.131355 30.167964 \r\n",
       "L 222.477501 36.324103 \r\n",
       "L 224.160183 45.169082 \r\n",
       "L 226.179403 57.178142 \r\n",
       "L 228.871695 74.849934 \r\n",
       "L 235.938962 122.393078 \r\n",
       "L 237.958181 134.173128 \r\n",
       "L 239.640864 142.776456 \r\n",
       "L 240.98701 148.710223 \r\n",
       "L 242.333156 153.698224 \r\n",
       "L 243.342766 156.770647 \r\n",
       "L 244.352375 159.239226 \r\n",
       "L 245.361985 161.082048 \r\n",
       "L 246.035058 161.954643 \r\n",
       "L 246.708131 162.538293 \r\n",
       "L 247.381204 162.830695 \r\n",
       "L 248.054277 162.830695 \r\n",
       "L 248.72735 162.538293 \r\n",
       "L 249.400423 161.954643 \r\n",
       "L 250.073496 161.082048 \r\n",
       "L 250.746569 159.923952 \r\n",
       "L 251.756179 157.661795 \r\n",
       "L 252.765788 154.787884 \r\n",
       "L 253.775398 151.327726 \r\n",
       "L 255.121544 145.856045 \r\n",
       "L 256.46769 139.48361 \r\n",
       "L 258.150373 130.405708 \r\n",
       "L 260.169592 118.17949 \r\n",
       "L 262.861884 100.33606 \r\n",
       "L 269.256078 57.178142 \r\n",
       "L 271.275297 45.169082 \r\n",
       "L 272.95798 36.324103 \r\n",
       "L 274.304126 30.167964 \r\n",
       "L 275.650272 24.935541 \r\n",
       "L 276.996418 20.709353 \r\n",
       "L 278.006028 18.240774 \r\n",
       "L 279.015638 16.397952 \r\n",
       "L 279.688711 15.525357 \r\n",
       "L 280.361784 14.941707 \r\n",
       "L 281.034857 14.649305 \r\n",
       "L 281.70793 14.649305 \r\n",
       "L 282.381003 14.941707 \r\n",
       "L 283.054076 15.525357 \r\n",
       "L 283.727149 16.397952 \r\n",
       "L 284.400222 17.556048 \r\n",
       "L 285.409832 19.818205 \r\n",
       "L 286.419441 22.692116 \r\n",
       "L 287.429051 26.152274 \r\n",
       "L 288.775197 31.623955 \r\n",
       "L 290.121343 37.99639 \r\n",
       "L 291.804026 47.074292 \r\n",
       "L 293.823245 59.30051 \r\n",
       "L 296.515537 77.14394 \r\n",
       "L 302.909731 120.301858 \r\n",
       "L 304.92895 132.310918 \r\n",
       "L 306.611633 141.155897 \r\n",
       "L 307.957779 147.312036 \r\n",
       "L 309.303925 152.544459 \r\n",
       "L 310.650071 156.770647 \r\n",
       "L 311.659681 159.239226 \r\n",
       "L 312.66929 161.082048 \r\n",
       "L 313.342363 161.954643 \r\n",
       "L 314.015437 162.538293 \r\n",
       "L 314.68851 162.830695 \r\n",
       "L 315.361583 162.830695 \r\n",
       "L 316.034656 162.538293 \r\n",
       "L 316.707729 161.954643 \r\n",
       "L 317.380802 161.082048 \r\n",
       "L 318.053875 159.923952 \r\n",
       "L 319.063484 157.661795 \r\n",
       "L 320.073094 154.787884 \r\n",
       "L 321.082704 151.327726 \r\n",
       "L 322.42885 145.856045 \r\n",
       "L 323.774996 139.48361 \r\n",
       "L 325.457678 130.405708 \r\n",
       "L 327.476898 118.17949 \r\n",
       "L 330.16919 100.33606 \r\n",
       "L 336.563384 57.178142 \r\n",
       "L 338.582603 45.169082 \r\n",
       "L 340.265286 36.324103 \r\n",
       "L 341.611432 30.167964 \r\n",
       "L 342.957578 24.935541 \r\n",
       "L 344.303724 20.709353 \r\n",
       "L 345.313334 18.240774 \r\n",
       "L 346.322943 16.397952 \r\n",
       "L 346.996016 15.525357 \r\n",
       "L 347.669089 14.941707 \r\n",
       "L 348.342162 14.649305 \r\n",
       "L 348.342162 14.649305 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_20\">\r\n",
       "    <path clip-path=\"url(#p884e1a932c)\" d=\"M 230.890914 88.74 \r\n",
       "L 315.025046 162.867273 \r\n",
       "\" style=\"fill:none;stroke:#800080;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_8\">\r\n",
       "    <path d=\"M 207.350184 170.28 \r\n",
       "L 207.350184 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_9\">\r\n",
       "    <path d=\"M 355.056066 170.28 \r\n",
       "L 355.056066 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_10\">\r\n",
       "    <path d=\"M 207.350184 170.28 \r\n",
       "L 355.056066 170.28 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_11\">\r\n",
       "    <path d=\"M 207.350184 7.2 \r\n",
       "L 355.056066 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       "  <g id=\"axes_3\">\r\n",
       "   <g id=\"patch_12\">\r\n",
       "    <path d=\"M 384.597243 170.28 \r\n",
       "L 532.303125 170.28 \r\n",
       "L 532.303125 7.2 \r\n",
       "L 384.597243 7.2 \r\n",
       "z\r\n",
       "\" style=\"fill:#ffffff;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_5\">\r\n",
       "    <g id=\"xtick_7\">\r\n",
       "     <g id=\"line2d_21\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"391.311146\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_17\">\r\n",
       "      <!-- −2 -->\r\n",
       "      <g transform=\"translate(383.940053 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-8722\"/>\r\n",
       "       <use x=\"83.789062\" xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_8\">\r\n",
       "     <g id=\"line2d_22\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"458.618452\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_18\">\r\n",
       "      <!-- 0 -->\r\n",
       "      <g transform=\"translate(455.437202 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"xtick_9\">\r\n",
       "     <g id=\"line2d_23\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"525.925758\" xlink:href=\"#m9899e5487b\" y=\"170.28\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_19\">\r\n",
       "      <!-- 2 -->\r\n",
       "      <g transform=\"translate(522.744508 184.878438)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"matplotlib.axis_6\">\r\n",
       "    <g id=\"ytick_11\">\r\n",
       "     <g id=\"line2d_24\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"384.597243\" xlink:href=\"#m259f5d8728\" y=\"154.485256\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_20\">\r\n",
       "      <!-- 0.5 -->\r\n",
       "      <g transform=\"translate(361.694118 158.284475)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_12\">\r\n",
       "     <g id=\"line2d_25\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"384.597243\" xlink:href=\"#m259f5d8728\" y=\"122.764164\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_21\">\r\n",
       "      <!-- 1.0 -->\r\n",
       "      <g transform=\"translate(361.694118 126.563382)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_13\">\r\n",
       "     <g id=\"line2d_26\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"384.597243\" xlink:href=\"#m259f5d8728\" y=\"91.043071\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_22\">\r\n",
       "      <!-- 1.5 -->\r\n",
       "      <g transform=\"translate(361.694118 94.84229)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-49\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_14\">\r\n",
       "     <g id=\"line2d_27\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"384.597243\" xlink:href=\"#m259f5d8728\" y=\"59.321979\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_23\">\r\n",
       "      <!-- 2.0 -->\r\n",
       "      <g transform=\"translate(361.694118 63.121198)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "    <g id=\"ytick_15\">\r\n",
       "     <g id=\"line2d_28\">\r\n",
       "      <g>\r\n",
       "       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"384.597243\" xlink:href=\"#m259f5d8728\" y=\"27.600887\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "     <g id=\"text_24\">\r\n",
       "      <!-- 2.5 -->\r\n",
       "      <g transform=\"translate(361.694118 31.400105)scale(0.1 -0.1)\">\r\n",
       "       <use xlink:href=\"#DejaVuSans-50\"/>\r\n",
       "       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n",
       "       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n",
       "      </g>\r\n",
       "     </g>\r\n",
       "    </g>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_29\">\r\n",
       "    <path clip-path=\"url(#pbcb452f51a)\" d=\"M 391.311146 162.867273 \r\n",
       "L 398.041877 160.412681 \r\n",
       "L 404.436071 157.842113 \r\n",
       "L 410.493729 155.170932 \r\n",
       "L 416.214849 152.417561 \r\n",
       "L 421.93597 149.419919 \r\n",
       "L 427.320555 146.356085 \r\n",
       "L 432.705139 143.037073 \r\n",
       "L 437.753187 139.674871 \r\n",
       "L 442.801235 136.050806 \r\n",
       "L 447.849283 132.144485 \r\n",
       "L 452.560795 128.224557 \r\n",
       "L 457.272306 124.020403 \r\n",
       "L 461.983817 119.511413 \r\n",
       "L 466.695329 114.675485 \r\n",
       "L 471.070304 109.871542 \r\n",
       "L 475.445279 104.744971 \r\n",
       "L 479.820253 99.274104 \r\n",
       "L 484.195228 93.435819 \r\n",
       "L 488.570203 87.205441 \r\n",
       "L 492.945178 80.556637 \r\n",
       "L 496.983616 74.023623 \r\n",
       "L 501.022055 67.086631 \r\n",
       "L 505.060493 59.720679 \r\n",
       "L 509.098931 51.899242 \r\n",
       "L 513.13737 43.594155 \r\n",
       "L 517.175808 34.775509 \r\n",
       "L 521.214246 25.411549 \r\n",
       "L 525.252685 15.468554 \r\n",
       "L 525.589221 14.612727 \r\n",
       "L 525.589221 14.612727 \r\n",
       "\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"line2d_30\">\r\n",
       "    <path clip-path=\"url(#pbcb452f51a)\" d=\"M 408.137973 156.238382 \r\n",
       "L 492.272105 81.607869 \r\n",
       "\" style=\"fill:none;stroke:#800080;stroke-dasharray:5.55,2.4;stroke-dashoffset:0;stroke-width:1.5;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_13\">\r\n",
       "    <path d=\"M 384.597243 170.28 \r\n",
       "L 384.597243 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_14\">\r\n",
       "    <path d=\"M 532.303125 170.28 \r\n",
       "L 532.303125 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_15\">\r\n",
       "    <path d=\"M 384.597243 170.28 \r\n",
       "L 532.303125 170.28 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "   <g id=\"patch_16\">\r\n",
       "    <path d=\"M 384.597243 7.2 \r\n",
       "L 532.303125 7.2 \r\n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n",
       "   </g>\r\n",
       "  </g>\r\n",
       " </g>\r\n",
       " <defs>\r\n",
       "  <clipPath id=\"p50ba7a7102\">\r\n",
       "   <rect height=\"163.08\" width=\"147.705882\" x=\"30.103125\" y=\"7.2\"/>\r\n",
       "  </clipPath>\r\n",
       "  <clipPath id=\"p884e1a932c\">\r\n",
       "   <rect height=\"163.08\" width=\"147.705882\" x=\"207.350184\" y=\"7.2\"/>\r\n",
       "  </clipPath>\r\n",
       "  <clipPath id=\"pbcb452f51a\">\r\n",
       "   <rect height=\"163.08\" width=\"147.705882\" x=\"384.597243\" y=\"7.2\"/>\r\n",
       "  </clipPath>\r\n",
       " </defs>\r\n",
       "</svg>\r\n"
      ],
      "text/plain": [
       "<Figure size 648x216 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def f(x):\n",
    "    return 0.5 * x**2  # Convex\n",
    "\n",
    "def g(x):\n",
    "    return np.cos(np.pi * x)  # Nonconvex\n",
    "\n",
    "def h(x):\n",
    "    return np.exp(0.5 * x)  # Convex\n",
    "\n",
    "x, segment = np.arange(-2, 2, 0.01), np.array([-1.5, 1])\n",
    "d2l.use_svg_display()\n",
    "_, axes = d2l.plt.subplots(1, 3, figsize=(9, 3))\n",
    "\n",
    "for ax, func in zip(axes, [f, g, h]):\n",
    "    ax.plot(x, func(x))\n",
    "    ax.plot(segment, func(segment),'--', color=\"purple\")\n",
    "    # d2l.plt.plot([x, segment], [func(x), func(segment)], axes=ax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_dgq9fj8",
    "id": "BFEAFAA06C6248D78969AA592C07BAB8",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### Jensen 不等式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_4k335lw",
    "id": "7BA75D0A48C84E1EBBE89BA3AD2326C3",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "\n",
    "$$\n",
    "\\sum_{i} \\alpha_{i} f\\left(x_{i}\\right) \\geq f\\left(\\sum_{i} \\alpha_{i} x_{i}\\right) \\text { and } E_{x}[f(x)] \\geq f\\left(E_{x}[x]\\right)\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_lya7oiy",
    "id": "C7382E20BBE0484082B1D60DC75F9170",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_mwk1azh",
    "id": "C9FEFF55DEE94C888B6BF96DA5F6EF30",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 性质\n",
    "1. 无局部极小值\n",
    "2. 与凸集的关系\n",
    "3. 二阶条件"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_m1mtghr",
    "id": "BF82E8C9E78C48558845FDC2D81D865F",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "###  无局部最小值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_8ops7to",
    "id": "84BF67FD09564B5B8186790E374E565C",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "证明：假设存在 $x \\in X$ 是局部最小值，则存在全局最小值 $x' \\in X$, 使得 $f(x) > f(x')$, 则对 $\\lambda \\in(0,1]$:\n",
    "\n",
    "$$\n",
    "f(x)>\\lambda f(x)+(1-\\lambda) f(x^{\\prime}) \\geq f(\\lambda x+(1-\\lambda) x^{\\prime})\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_12pny60",
    "id": "7CD6765AEB59486E8614F41B02FA4356",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 与凸集的关系"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_zt6ghvy",
    "id": "EA4449CCC83843F689B7A8C841670CDB",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "对于凸函数 $f(x)$，定义集合 $S_{b}:=\\{x | x \\in X \\text { and } f(x) \\leq b\\}$，则集合 $S_b$ 为凸集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_7ie79v4",
    "id": "5E225FED3D9942A8B2C51993678D87BB",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "证明：对于点 $x,x' \\in S_b$, 有 $f\\left(\\lambda x+(1-\\lambda) x^{\\prime}\\right) \\leq \\lambda f(x)+(1-\\lambda) f\\left(x^{\\prime}\\right) \\leq b$, 故 $\\lambda x+(1-\\lambda) x^{\\prime} \\in S_{b}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_xnx5dmx",
    "id": "48996626B2B34EE18942B43A611CA092",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "$f(x, y)=0.5 x^{2}+\\cos (2 \\pi y)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "graffitiCellId": "id_vxbtgtf",
    "id": "F55A64B60E0743CEA4945D3AA29C6993",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"https://cdn.kesci.com/rt_upload/F55A64B60E0743CEA4945D3AA29C6993/q5p1wzmavw.svg\">"
      ],
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x, y = np.meshgrid(np.linspace(-1, 1, 101), np.linspace(-1, 1, 101),\n",
    "                   indexing='ij')\n",
    "\n",
    "z = x**2 + 0.5 * np.cos(2 * np.pi * y)\n",
    "\n",
    "# Plot the 3D surface\n",
    "d2l.set_figsize((6, 4))\n",
    "ax = d2l.plt.figure().add_subplot(111, projection='3d')\n",
    "ax.plot_wireframe(x, y, z, **{'rstride': 10, 'cstride': 10})\n",
    "ax.contour(x, y, z, offset=-1)\n",
    "ax.set_zlim(-1, 1.5)\n",
    "\n",
    "# Adjust labels\n",
    "for func in [d2l.plt.xticks, d2l.plt.yticks, ax.set_zticks]:\n",
    "    func([-1, 0, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_t65qx4z",
    "id": "9F6DB0922B9E437B83607C649DC90A39",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 凸函数与二阶导数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_8khofr0",
    "id": "7DD8D13EE2B5473B89D02945E812F86B",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "$f^{''}(x) \\ge 0 \\Longleftrightarrow f(x)$ 是凸函数\n",
    "\n",
    "**必要性 ($\\Leftarrow$):**\n",
    "\n",
    "对于凸函数：\n",
    "\n",
    "$$\n",
    "\\frac{1}{2} f(x+\\epsilon)+\\frac{1}{2} f(x-\\epsilon) \\geq f\\left(\\frac{x+\\epsilon}{2}+\\frac{x-\\epsilon}{2}\\right)=f(x)\n",
    "$$\n",
    "\n",
    "故:\n",
    "\n",
    "$$\n",
    "f^{\\prime \\prime}(x)=\\lim _{\\varepsilon \\rightarrow 0} \\frac{\\frac{f(x+\\epsilon) - f(x)}{\\epsilon}-\\frac{f(x) - f(x-\\epsilon)}{\\epsilon}}{\\epsilon}\n",
    "$$\n",
    "\n",
    "\n",
    "$$\n",
    "f^{\\prime \\prime}(x)=\\lim _{\\varepsilon \\rightarrow 0} \\frac{f(x+\\epsilon)+f(x-\\epsilon)-2 f(x)}{\\epsilon^{2}} \\geq 0\n",
    "$$\n",
    "\n",
    "\n",
    "**充分性 ($\\Rightarrow$):**\n",
    "\n",
    "令 $a < x < b$ 为 $f(x)$ 上的三个点，由拉格朗日中值定理:\n",
    "\n",
    "$$\n",
    "\\begin{array}{l}{f(x)-f(a)=(x-a) f^{\\prime}(\\alpha) \\text { for some } \\alpha \\in[a, x] \\text { and }} \\\\ {f(b)-f(x)=(b-x) f^{\\prime}(\\beta) \\text { for some } \\beta \\in[x, b]}\\end{array}\n",
    "$$\n",
    "\n",
    "\n",
    "根据单调性，有 $f^{\\prime}(\\beta) \\geq f^{\\prime}(\\alpha)$, 故:\n",
    "\n",
    "$$\n",
    "\\begin{aligned} f(b)-f(a) &=f(b)-f(x)+f(x)-f(a) \\\\ &=(b-x) f^{\\prime}(\\beta)+(x-a) f^{\\prime}(\\alpha) \\\\ & \\geq(b-a) f^{\\prime}(\\alpha) \\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "graffitiCellId": "id_fb4h7jk",
    "id": "1084E24A125F422180CC5907909885E1",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(-1.5, 0.125, 'x')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"https://cdn.kesci.com/rt_upload/1084E24A125F422180CC5907909885E1/q5p21ulbo9.svg\">"
      ],
      "text/plain": [
       "<Figure size 252x180 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def f(x):\n",
    "    return 0.5 * x**2\n",
    "\n",
    "x = np.arange(-2, 2, 0.01)\n",
    "axb, ab = np.array([-1.5, -0.5, 1]), np.array([-1.5, 1])\n",
    "\n",
    "d2l.set_figsize((3.5, 2.5))\n",
    "fig_x, = d2l.plt.plot(x, f(x))\n",
    "fig_axb, = d2l.plt.plot(axb, f(axb), '-.',color=\"purple\")\n",
    "fig_ab, = d2l.plt.plot(ab, f(ab),'g-.')\n",
    "\n",
    "fig_x.axes.annotate('a', (-1.5, f(-1.5)), (-1.5, 1.5),arrowprops=dict(arrowstyle='->'))\n",
    "fig_x.axes.annotate('b', (1, f(1)), (1, 1.5),arrowprops=dict(arrowstyle='->'))\n",
    "fig_x.axes.annotate('x', (-0.5, f(-0.5)), (-1.5, f(-0.5)),arrowprops=dict(arrowstyle='->'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_dcklggv",
    "id": "1DD74B10939F4E39966F47334D54DA74",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 限制条件"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_mzhwoq7",
    "id": "EA74092CABA74777B4406FDB8E64E5A3",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "\n",
    "$$\n",
    "\\begin{array}{l}{\\underset{\\mathbf{x}}{\\operatorname{minimize}} f(\\mathbf{x})} \\\\ {\\text { subject to } c_{i}(\\mathbf{x}) \\leq 0 \\text { for all } i \\in\\{1, \\ldots, N\\}}\\end{array}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_of1p828",
    "id": "23F75693A8FA47E29BDF3ADB99FCBF12",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 拉格朗日乘子法\n",
    "[Boyd & Vandenberghe, 2004](https://d2l.ai/chapter_references/zreferences.html#boyd-vandenberghe-2004)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_lrmo53z",
    "id": "5410AD20966D4FDE856D8BD4B4941DB3",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "\n",
    "$$\n",
    "L(\\mathbf{x}, \\alpha)=f(\\mathbf{x})+\\sum_{i} \\alpha_{i} c_{i}(\\mathbf{x}) \\text { where } \\alpha_{i} \\geq 0\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_rd9dyl7",
    "id": "4FAA124C57D148758B488AE2B04DE2DE",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 惩罚项"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_40i7dc7",
    "id": "4CE61753947A4F84AB9CB2280BAFB6E4",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "欲使 $c_i(x) \\leq 0$, 将项 $\\alpha_ic_i(x)$ 加入目标函数，如多层感知机章节中的 $\\frac{\\lambda}{2} ||w||^2$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_0hnopfa",
    "id": "F61D5B7240444B2D8B5B829479CB9E9E",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 投影"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_3i4aa5i",
    "id": "F6A92112680E4E8B83B9024617509699",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "\n",
    "$$\n",
    "\\operatorname{Proj}_{X}(\\mathbf{x})=\\underset{\\mathbf{x}^{\\prime} \\in X}{\\operatorname{argmin}}\\left\\|\\mathbf{x}-\\mathbf{x}^{\\prime}\\right\\|_{2}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_xfur0e6",
    "id": "E551DDB7CD3149EB86C95B58671C2D3D",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "\n",
    "![Image Name](https://cdn.kesci.com/upload/image/q5p241skoc.png?imageView2/0/w/640/h/640)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
